The Fix Is In
When the results of last spring's Texas Assessment of Academic Skills were reported, little Kashmere Gardens Elementary was the star of the show in the Houston Independent School District. The nearly all-black inner-city school had outdone itself since the previous year, when only 54 percent of the school's fourth-graders had passed the reading portion of the all-important TAAS, and even fewer, 37 percent, passed in math, a chillingly low figure for any HISD principal.
The school made a robust effort to make sure not one of its 588 children fell through the cracks, according to Principal Margaret Jefferson, who later told the Houston Chronicle that even the school's custodians, crossing guards and food service personnel tutored students who needed extra help. Colorful reminders blared from the school's billboards and posters: "Soar High on TAAS."
When the scores came back, the news was almost too good to be true: Every single eligible fourth grader passed both the reading and math portions of the test, boosting the school from a rating of "acceptable" to "recognized." District dignitaries swooped down from their perches high in the HISD bureaucracy to congratulate the school. Not a visitor left the campus without a full briefing on how well Kashmere had done. Even the K-Ram cheerleaders aerialed themselves onto the bandwagon, yelling: "Kashmere Rams gonna bust that test! Swing it all around! Show who's the best!"
"Our success," Jefferson told the Chronicle, "was a deliberate plan."
Just how deliberate is cause for much speculation among Kashmere Gardens's past and current teachers, who asked not to be named for fear of retribution. Some of them suspect that school employees tampered with student answer forms to improve the school's scores, and statistical evidence lends support to what they say. Last year, one third-grade class at Kashmere had 15 times more erasures than the state average, and 100 percent of the answers were changed from wrong to right. Students who perform poorly in their classes and on other tests do mysteriously well on the TAAS. Yet despite the evidence, the question is daunting: Would a school really go so far as to outright cheat?
Although educators are not, on the whole, known to be a disreputable bunch, given the high stakes of the test it's a reasonable question. The party line about the pervasive emphasis on the TAAS is that it's "for the children," but there are other, more immediate reasons to make sure scores are high.
In the world of public education in Texas, no test -- not the SAT, not the final exam, not the weekly spelling quiz, not the entrance test for gifted and talented programs -- comes close to matching the Texas Assessment of Academic Skills in importance. The exit-level TAAS, given beginning in tenth grade, is the one test students must pass before graduating from high school.
Before high school, children take the test every year beginning in third grade, but those results are used to rate campuses and school districts, not students. Schools that consistently score low on the TAAS risk being taken over or even shut down by the state. In some districts, including HISD, principals whose schools don't do well enough can lose their jobs.
Performing well has its rewards. Teachers receive candy, flowers -- and pay bonuses. There are banquets and press conferences, banners and marching bands and, most important in terms of public perception, school ratings: "exemplary" or "recognized" for schools that show muscle, "acceptable" or "low-performing" for those that lag behind. Upper-level management shares in the bounty: HISD's Superintendent of Schools, Rod Paige, can receive a bonus of up to $25,000 based solely on the district's TAAS scores.
The kids, on the other hand, have to be brainwashed into caring, since the test doesn't particularly affect them. To that end, schools use "TAAS It Up!" pep rallies and motivational slogans, Saturday tutorials and endless practice tests, test-taking strategy books such as Breaking the Code and bribes of AstroWorld tickets, parties and special clubs. Much of this institutional hysteria is not, one assumes, what lawmakers envisioned when they put the test in place, but it's legal and some of it might even work.
On the other hand, one might suppose that outright cheating would be difficult to do. What with the TAAS's status as the be all and end all of Texas public education, the state might be expected to put massive roadblocks in the way of any would-be tamperer. But in fact, there is nothing short of integrity to stop a school from artificially raising scores, and the list of required equipment is short: All you need is a No. 2 pencil with a good eraser.
According to Kashmere Gardens Principal Margaret Jefferson, the Chronicle reported, the school's motto is "Striving for Exemplary Success." But one former teacher has another version of the Kashmere credo: "Do what you have to do." Instead of crossing guards or cafeteria workers tutoring students (which several teachers say they've never seen), employees describe tests administered with window shades pulled and doors locked, teachers not permitted to give the test to their own classes and poor students earning mysteriously high scores. (Jefferson declined to be interviewed by the Press.)
Two teachers who have moved from Kashmere Gardens to other schools say they were shocked when their normally struggling students received high marks on the TAAS. One says she had an overflow class of particularly low-performing students one year, yet almost all of them received "academic recognition" when their scores came back, even one whom she had tried to get tested for learning disabilities. "When I saw the test scores, I said, 'There's no way those kids passed.' I mean, they couldn't read."
She remembers that a veteran Kashmere Gardens faculty member told her not to worry if her students didn't seem like they could pass the test. "Just help them," the older teacher advised.
The younger teacher, who says her colleague was "a pointer" who would tell children to go back and rework problems they had gotten wrong, replied, "I am not helping them.... That's a disservice to those children because they can't read now, they're not going to read next year, and everybody thinks they can read."
Artificially boosting scores may be one way to draw favorable attention to a school, but it does no justice to children like the one whom the teacher tried to have tested for learning disabilities. His mother, who asked that neither she nor her son be named, says he passed the TAAS every year. Now, she's noticed her son has serious difficulty reading, and she's tutoring him every night to prepare for this year's TAAS.
Records obtained by the Press show that one fourth-grader at Kashmere who received high marks on last year's TAAS -- a 92 out of 94 on reading and an 83 out of 93 on math -- scored well below grade level on the Stanford Achievement Test and, this year, was in danger of failing courses.
Parent volunteer Julia Keller, who until recently helped out in the school library every day, says even she's heard rumors of test-tampering. "I've heard a lot of teachers around that school myself, personally, say that that's what [the school] was doing on these TAAS tests. I just said it was a shame." Keller recently withdrew her twin fifth-graders, Derric and Erric, to transfer them to another school.
Although neither of the former Kashmere Gardens teachers who spoke to the Press observed anyone tampering with tests, both left the school because they were convinced foul play had occurred. "I'm as sure as I can be without seeing [the answer forms] erased and changed," says one. "The scores were a joke."
The procedure for administering the TAAS offers no shortage of opportunities for those who want to "help" students pass. The sealed test booklets and answer forms arrive at a school a day or two before the test is scheduled to be given. The designated campus TAAS coordinator, usually the school counselor, signs for the tests and stores them in a secure place. On each day of testing, the coordinator passes out answer forms and booklets to each teacher in the morning and picks them up at the end of the testing period. The day after testing, the coordinator delivers the tests to HISD's testing center, and from there the tests are sent to Iowa City, Iowa, to be scored.
Overnight and after the test, the answer forms are in the control of the school administration. During those periods, the testing coordinator is permitted to darken light answers and erase stray marks on the answer forms to prevent the computer from misreading them.
The Kashmere teachers' suspicion of tampering is supported by statistical evidence in the form of a little-known and, until recently, little-used document called the erasure analysis. When tests are scored, the Texas Education Agency's testing contractor, National Computer Systems, scans each answer form for erasures and notes whether erased answers were changed from wrong to right, right to wrong or wrong to wrong. The resulting erasure analysis lists every classroom in the state whose kids had an especially high average numbers of erasures (the statewide average lies between zero and one erasure per child; the report lists classes with averages greater than three standard deviations from the mean).
A high number of erasures is not in itself evidence of cheating. The analysis tells you, for example, that in 1997 one of Scott Elementary's fifth-grade classes had a whopping eight erasures per child, and 96 percent of them went from wrong to right, but it doesn't tell you how or why. A class could have started at the wrong place on an answer form. Or one child who made a lot of mistakes could throw off the class average. Or an especially conscientious teacher could train her students to double-check their work. But mostly, those would be isolated incidents; a class showing up once on the erasure analysis is not cause for concern. What's interesting, however, is that some teachers show up year after year, and some entire schools show up much more frequently than others.
Over the past three years, 36 Kashmere Gardens classes appeared on the erasure analysis, more than any other HISD school, and 12 more than the school with the next-highest number of appearances, Key Middle School. With a 41-point improvement in math, only one other elementary school in the Northeast District, E.O. Smith, made gains that even came close to rivaling Kashmere Gardens's math scores. Most who improved did so by four to 12 percentage points. Despite its gains in reading and math, Kashmere's passing rate declined two percentage points on the writing test, which is more difficult to tamper with because it includes a written composition.
Furthermore, Kashmere's appearances on the erasure analysis correlate with the school's scores. In 1996, when the school was rated "exemplary," 16 classes show up on the erasure analysis. In 1997, when Kashmere Gardens librarian William Price says the school had been warned that the district would be closely monitoring its TAAS administration, there were five classes on the erasure analysis. The school's rating dropped two notches, to "acceptable." In 1998, when scores skyrocketed to nearly 100 percent passing, so did the number of classes appearing on the erasure analysis -- to 15.
Even with that kind of evidence, pinning down perpetrators isn't easy, as former HISD TAAS coordinator Carl Shaw knows only too well. Shaw oversaw HISD's department of student assessment from 1981 to 1995, when he left for a job as head of research and evaluation at Fort Worth ISD. Early on, Shaw advocated the state's accountability system, and he has served on numerous state committees dealing with assessment and the TAAS. Now he works for himself as an independent, Austin-based consultant who's developed assessments used in alternative education programs.
While Shaw, a bluntly spoken man with a hint of country twang, supports the idea of testing Texas students, he's disappointed in both the way the TAAS is structured and the way it's administered. He became an independent consultant because he grew weary of what he calls the "conspiracy" of an educational establishment which, he says, would find any way possible to conceal the truth about student performance. Cheating, he stresses, was only one of those ways.
"There's so dang much cheating in Houston," Shaw says, but the district's attitude about punishing cheaters was, he adds, "blase."
"Unless the perpetrator broke down and confessed and offered to resign, nothing ever happened," Shaw says. "I can think of one case where it was so blatant that we got involved and got all the little kids interviewed, and by the time we got it to a hearing a year later all the kids had forgotten about it, every one of them. It was kind of strange."
Although he declined to name any specific schools, Shaw remembers another case where, he believes, a school got ahold of the TAAS writing topic early and used it to coach students before the test. "The principal lied and they got by with it. They made up all kinds of bull crap. When you see this stuff, and you see the product of kids' efforts over the years, you know when you're being lied to and when you're not."
It was thanks to Shaw that National Computer Systems began doing the erasure analysis back in 1993 or 1994. Shaw suspected an HISD school (not Kashmere Gardens) of tampering, so he worked with the contractors to devise a way to screen answer forms. Although the analysis appeared to confirm his suspicions, Shaw says, nothing happened to the school; even if tampering is detected, there's often no way to determine who did it. Furthermore, Shaw admits, cheating investigations are not a task anyone is eager to undertake if they don't have to. In the case of the tampering, he says, "The teachers didn't know this had occurred, and they were so durn proud of what happened ... the pride was there before we figured out what was going on." In that case, Shaw says, the threat of investigation was enough: "As soon as the method [of tampering] got detected it kind of went away."
That might be the case again this year. When deputy superintendent of school administration Faye Bryant discovered the erasure analysis, she responded by notifying every campus TAAS coordinator of its existence. She warned in a memo that the district would more closely monitor schools "during and after" TAAS administrations. "With the growing accountability attached to TAAS scores, it is essential that you maintain the highest integrity during the test," Bryant wrote.
Bryant may not have been aware of the analysis before this year because the district doesn't put it to regular use. In fact, HISD hadn't even noticed a stark fact that begs for explanation: while many of Texas's 1,042 school districts never register excessive erasures, a full 31 percent of the classes that show up on the statewide erasure analysis for the past three years are from HISD, which has only 5 percent of the state's students. Furthermore, erasures become a problem when school ratings are involved: Over the past three years, 71 HISD eighth-grade classes have shown up on the reading, writing and math portions of the TAAS, while only 14 have shown up on the science and social studies portions, which are not included in accountability ratings.
Asked to explain the district's high occurrence of erasures, Chief of Staff for Instructional Services Susan Sclafani, the district's second-in-command, said, "We were only concerned about what was happening in our district, so I didn't even look at how the numbers compared."
The district also hadn't noticed that of the 29 schools with more than five classes showing up on the analysis in the past three years, 26 of them are in one of three sub-districts: the Northeast, headed before this year by Superintendent Charles Herbert; South Central, headed before this year by Superintendent Andre Hornsby; or South, headed by Larry Alexander. The numbers don't surprise American Federation of Teachers president Gayle Fallon. Those districts, she notes, have higher levels of poverty and more low-performing schools, schools that might have difficulty coping with cranked-up pressure to pass the TAAS. Beyond that, Fallon cites a fear-based management style. "Whenever we hear something negative," she says, "South and South Central show up." Fallon says the supers in those districts had an "autocratic, my-way-or-the-highway" leadership style. "Consequently," she says, "We found principals that were really afraid of their bosses."
District spokesman Terry Abbott says it's possible such statistical clustering is a "coincidence."
"HISD is 25 percent larger than the next largest school district in Texas," Abbott says. "Is it conceivable that we could make up 30 percent of the erasure report? Sure. That doesn't mean there's a problem there....In order to determine if there's a problem there, we have to investigate. And that is what we're doing." Last July, the district undertook an investigation of eight schools -- Kashmere Gardens, Frost, Gregg, Hobby, Montgomery and Scott elementaries and Key and Cullen middle schools -- in a massive operation that some have taken to calling Erasergate.
An initial review of the erasure data, Abbott points out, "gives HISD reason to further investigate the scores from only 56 classes -- or less than one-half of one percent" of HISD's total. But according to district documents, the 56 classes aren't the only ones with problems, they're just the first group the district chose to examine.
Erasergate began after the 1998 TAAS, when HISD received "an anonymous tip that some teachers had improperly aided test takers," according to a July press release. Abbott says that tip prompted Paige's office to request the past five years of erasure data from the Texas Education Agency. After screening out the less egregious classes (such as those where less than 75 percent of the erasures went from wrong to right), the district identified 32 schools from 1998 alone that warranted a closer look, according to a draft memo by Manager of Student Assessment Sandra Kriendler, who declined to be interviewed by the Press.
Because reviewing individual answer forms and test documents would take so much time, Kriendler recommended that only the ten "most suspect" schools be investigated first (ultimately, time constraints reduced that to eight). From there, the district could decide "whether or not it wishes to invest more time and effort on the remaining schools." Kriendler's office has turned over the investigation of the first eight schools to the district's Professional Standards department, which is interviewing the TAAS coordinator and other employees at each school.
Teachers' union officials worry that the district will peg teachers for cheating, when -- they say -- principals are the ones with more incentive and opportunity to tamper with tests. When asked, Chief of Staff Sclafani insists the district is investigating "campuses," but HISD does appear predisposed to the idea that Erasergate will unearth only one type of guilty party. "If the investigators have reason to believe there was indeed wrongdoing," a district press release on the investigation says, "... the teachers will be banned from administering the test."
Lack of trust is one reason investigators may come up empty-handed when they try to interview employees. Because principals have been given so much power, Fallon says, employees -- particularly office aides who are likely to witness cheating but have less job protection than teachers -- are often afraid to speak up. "When it comes right down to it," says Jon Dansby, president of the Houston Educators Association, "Nobody's going to back them up."
Shaw agrees. "The fear factor in the district keeps people paralyzed," he says.
If you ask teacher Ada Johnson, a former counselor at Scott Elementary, she'll tell you she had a "wonderful relationship" with her boss, Principal Artice Hedgemon, until she tried to report cheating on the TAAS in 1996. In an administrative hearing, Johnson testified that she saw teachers pointing out answers and correcting children's work on the TAAS and informed the principal of the matter. When Johnson, who said she was so close to Hedgemon she served as the "unofficial" assistant principal, followed up on her allegations, Hedgemon became "very cold and very distant," Johnson testified. The next month, Hedgemon told Johnson that her counseling position was being cut.
Johnson, a district employee since 1970 whose job performance as a counselor had been evaluated at 4.3 to 4.66 on a 5-point scale, spent the next two years "awaiting assignment" in the lounge of the Northeast District office. During that period, the district filled 59 counselor positions.
Finally, Johnson, who taught English, a TAAS subject, when she was a classroom teacher, was assigned to teach high school social studies, a position where, she noted, "I would not be around the TAAS-taking procedure." The board of trustees, saying Johnson offered no hard evidence that she had attempted to report cheating on the TAAS test, denied her request to be assigned as a counselor, a more lucrative position than teacher. Deputy superintendent Bryant told the Press that "Ada Johnson has a long history of problems with the district" but refused to elaborate.
The district is now investigating Scott Elementary.
Ultimately, the district depends on those who witness impropriety to come forward; if HISD doesn't hear about cheating, it can't investigate it. But cases like Johnson's appear to support what many say about HISD: The district cultivates a hear-no-evil, see-no-evil culture, and there's no reward for those who want to call attention to wrongdoing. "The retaliation is not subtle," says union president Gayle Fallon. "It's blatant."
Yet just as cheating can be difficult to prove, so can retaliation. Often, as with Johnson's case, it's one person's word against another. The administrator is the one who's likely to be believed, Fallon says.
Fallon says her union once had a case where a principal directed teachers' aides to alter a test and, fearing the loss of their jobs, they complied. When they tried to come forward, Fallon asked the district to grant them immunity, and the district refused. Without protection, Fallon advised the aides not to report what they had seen. Houston Educators Association president Dansby says his union has been in similar situations. "If Governor Bush granted amnesty, his phone would be ringing off the hook," Dansby says.
Chief of Staff Sclafani contends it's difficult to protect someone who has participated in improper handling of the TAAS. "Somebody who comes to us in advance we certainly can protect," she says. "Somebody who participates in cheating, it's a little difficult to say that they shouldn't be held accountable. I think every teacher understands that the maintenance of their certification is dependent upon their ethical behavior, directive or no directive."
Nonetheless, Sclafani says, the district has tried to alleviate some of the fear of retaliation by implementing a whistle-blower policy and installing a hot line to take anonymous complaints. "We know that in some cases employees don't have full confidence in the process that will be used. And we want them to -- we want them to know that this district stands for what's right. And we need them to stand with us on what's right. And we've given them an anonymous way to bring the information forward so that we can deal with the action and make sure that nobody is working in an environment that is less than totally ethical and legal."
Still, the perception persists that the district does not welcome complaints. Fallon says she tells union members not to use the hot line because she doesn't trust the district to keep the information private. In a termination hearing that Sclafani presided over, M.C. Williams Principal Roy Morgan testified that his supervisor showed him a hot line complaint accusing Morgan of abusing students. The principal said the hot line transmittal noted when and from where the call was made, and from that he was able to deduce -- correctly -- who had made the call.
Abbott says hot line transmittals don't record where a call emanates from, adding that the principal was most likely confused about what he saw on the form.
Still, principals in HISD have a virtual fiefdom when it comes to who works for them. They can transfer, or even fire, employees, who then have to fight their way through the system. "[Employees have] all these procedures to turn things in," says Fallon. "Of course, their life is hell after they do it. Heads, they're screwed. Tails, they're screwed."
Sometimes allegations of cheating simply disappear between the district's layers of bureaucracy. Suzanne Ely, who taught at Cornelius Elementary in the South Central District in the 1996-97 school year, reportedly told the district of cheating. Ely, now a graduate student at Harvard Divinity School, confirmed that she made such allegations but declined to give details, adding that the district took no action. According to Abbott, the district has no record of any communication from Ely, and an open records request turned up no allegations concerning Cornelius Elementary.
One of the teachers who left Kashmere Gardens also reported cheating beginning in 1994 -- twice to Carl Shaw, whom the teacher says advised a transfer to a different school (Shaw says while he doesn't remember the exact incident, under certain circumstances, such as a teacher unwilling to make a signed allegation, he might have told a teacher to transfer), and once to an official in the Northeast District office. "The district knew about it," the teacher says.
Bryant insists the district vigilantly pursues any hint of suspicion. "Any time we give TAAS or any other test, if there are rumors, there are allegations, we must follow up and check everyone if it takes us a year or two," Bryant says. "It's our obligation, and that's what we sign our name to."
Last May, North District Superintendent Erasmo Teran received a letter alleging that TAAS booklets at one of his schools were tampered with. The letter was signed "Berry Elementary Staff." Subsequent signed letters from individual teachers complained that, among other things, Berry Principal Graciela Kavulla had announced that the school was "exemplary" shortly after the test was administered (but before it was scored), and that campus TAAS coordinator Patricia Morgan had been seen going through test booklets with a pencil in hand, an activity that's bound to look suspicious to those who don't know that coordinators can remove stray marks.
The investigation documents provide an inside view of how schools handle the TAAS, and of how HISD handles its investigations.
Last year, Berry's TAAS performance jumped from 57 percent of kids passing reading, writing and math to 94.5 percent passing, a leap Kavulla attributed to an aggressive plan she developed over the summer of 1997. In her interview with Mike Martin, who investigated the case, Kavulla said the school closely monitored each child's progress throughout the year. In a school as small as Berry, Kavulla succinctly pointed out, "Seven, ten kids can knock it out for you."
Kavulla's plan focused on using test-taking "strategies," such as underlining key words and drawing arrows when rounding numbers. Children who used the strategies successfully on practice tests were rewarded with, among other things, movies and popcorn. They were told if they used the strategies on the TAAS, they would get a trip to AstroWorld.
That wasn't all they got. One teacher interviewed by Martin, Kristy Mikus, said Kavulla gave teachers candy and instructed them to give it out during the TAAS test to kids who were using the strategies. If a child was not using the strategies, Mikus said, Kavulla told her to whisper "Use the strategies" to the child during the test. Mikus also said Kavulla encouraged her to send low-performing kids to the nurse if they weren't feeling well.
Such practices, however questionable, were minor points compared to the question of whether or not children's answers had been changed after the fact. As campus TAAS coordinator, Morgan was responsible for distributing and collecting the tests. She told investigators that several office aides and one teacher helped her erase stray marks from the answer forms and said she checked over every test booklet to make sure that kids were using the strategies. She and the other employees who "cleaned up" answer forms all denied changing any students' answers or seeing anyone changing answers. A review of the answer forms, according to Martin's report, uncovered no evidence that tests had been tampered with.
However, interview transcripts reveal a few troubling details that never made it into Martin's report. One was Mikus's description of rewarding kids with candy during the test. Another was that an office aide who inspected answer forms, Mirna Garcia, told Martin she was told to put aside those where students had left questions blank.
Even more startling, Morgan was permitted to keep the answer forms for three extra days. "I always turned them in late and there's never been a problem," Morgan told the investigator. "They've always said, 'Just get them there as soon as you can, as soon as you've finished what you have to do.' "
Her admission was not enough to alarm the district. When asked about it, Teran issued the ultimate see-no, hear-no response: "There is no information that the school regularly turned in the test late."
Still another odd finding raised no official eyebrows: Morgan made her own answer key for the test. "Ms. Morgan denied that she took the TAAS but acknowledged that she did write down several question numbers and corresponding answer letters on a yellow tablet," the report says. "Ms. Morgan also acknowledged that she did in fact use the yellow tablet to compare several students [sic] answers." As to why Morgan wanted to compare students' answers, no explanation is offered.
Still, the district had decided to investigate Kavulla, not Morgan, and Martin said he could not confirm charges that Kavulla violated testing regulations. As far as Morgan and her assistants' reviewing test forms to see how children performed and checking booklets to see if children used strategies, Martin wrote that "one can interpret" such activities as a violation of TAAS regulations, which state, "There must be no unauthorized viewing of the contents of test booklets and answer documents" and "No person may review student responses without specific permission."
The district insists the investigation uncovered no evidence of wrongdoing.
In still another case, HISD decided there was no problem without even investigating. In the spring of 1997, an anonymous call prompted Sandra Kriendler's office to review erasure data on Kashmere High School. Kriendler's office reported that there were "a significant number" of students who had more than 75 percent of their erasures going from wrong to right. Furthermore, the report noted, "Several test booklets were marked with the original answer choice, yet were not marked with the change in answer choice. Others showed the change, but whereas the original marking was in marker or pencil, the change was made in a different medium. This suggests that someone other than the student made the change."
Kriendler recommended that Professional Standards and Student Assessment interview staff and students, that students' test scores be compared with their previous attempts to pass the exit-level exam, and that the district "should give the TEA periodic updates on the progress of our investigation."
The district did none of those things. An open records request turned up nothing further about Kashmere High. Asked to explain, Abbott provided a somewhat squirmy account of events.
First, Abbott said the area superintendent for the Northeast District, who at that time was Charles Herbert, had his staff readminister the TAAS to the Kashmere students. "The state provided an alternate test," Abbott said. "The kids did better, many of them, than on the original test. Obviously, this was the best way to handle the situation. If there was some sort of impropriety with the original testing, certainly a new test would answer that question."
When TEA senior director of assessment Keith Cruse said he had no knowledge of alternate tests being issued, the Press questioned Abbott again. Abbott queried district personnel and relayed that the district had used a previously released TAAS (the state releases them after they're given) and scored the tests in-house. "Using a released test made sure that it wasn't the same test [the students took before]," Abbott said.
Generally speaking, this sort of retesting would put any doubts about test security to rest. But in order to get valid results the test must be readministered quickly -- within four weeks of the original test. "If you went six weeks or longer you would expect increased scores," Cruse says.
Pressed further, Abbott again queried, and again relayed that the tests had been readministered the following January, eight months later. By then the 1997 TAAS would have been released, so how did the district make sure the kids weren't taking the exact same test they took in the spring?
Press: Wait a minute, the math scores went down?
Abbott: "On math, 133 were tested the first time and 88.4 percent passed. On the retest, 105 were tested, but only 69 of the original kids who took the test. On the math retest, the passing rate was 56 percent."
Okay, bear with us for a minute while we review: Kriendler found troubling evidence of tampering on Kashmere's TAAS tests. Students -- a substantially different group of students -- took the same test again, eight months later, and the number who passed math dropped more than 30 percentage points. From this, the district concluded that there was no substance to the allegations that someone, or many someones, at Kashmere High cheated.
The Texas Education Agency has very little to say about the district's foot-dragging. In the opinion of those who work at TEA, HISD is problem-free and the TAAS is adequately secure. After all, as the agency's associate commissioner for curriculum, assessment and technology Ann Smisko is quick to point out, every educator who handles the test signs a security oath.
Signed oaths, much like group prayer or homeopathic medicines, might have some beneficial effect, but it's hard to argue that they function materially to prevent cheating. On the other hand, they're cheap. More concrete security measures such as outside proctors, tamper-proof baggies for answer forms, or extra seals on booklets cost too much, the agency claims. "Just simple things like that just increase the price tremendously," says Joe Lucio, head of the TEA's TAAS Security Task Force. "We're testing over two million kids per year, so that's a lot of money if you put something per student."
On the local level, too, resources are scarce, HISD has so little manpower to investigate cheating, the district won't even venture a guess as to when Erasergate will be completed. And although some Texas districts have implemented their own safeguards, such as picking up the test from schools every night, HISD has yet to follow their lead. "There is no test that can be secured," Fallon says, "if the administration doesn't want it to be."
Since the Texas Education Agency doesn't put resources into test security on the front end, one might assume they'd be extra vigilant on the back end. But while the agency has a special data unit that systematically scrutinizes "red-flag" statistics, such as absences or exemptions for children who are in special ed or have limited English skills, to see if they're higher than normal, no one at the agency systematically reviews the erasure analysis. In interviews conducted in December and January, four administrators --Smisko, Lucio, director of assessment Keith Cruse and Kay Thomas, executive assistant to the associate commissioner for accountability Linda Mora -- told the Press that they do not use the analysis.
"There just haven't been so many [reports] that we have determined to be real cheating that we felt the need to make a change in the [testing] procedure," Smisko said.
On January 22, however, the agency issued letters to 11 school districts including the Houston-area HISD, Fort Bend and North Forest, asking them to investigate excessive erasures. "It's the first time we've sent out a letter like that," says TEA spokeswoman Debbie Ratcliffe. The letter, signed by Commissioner of Education Mike Moses, was drafted by Cruse, Mora and Smisko, Ratcliffe says.
The agency asked HISD to investigate 15 schools (including seven of the Erasergate eight) by April 1. Abbott points out that this is 5.2 percent of HISD's schools, while in the smaller districts, Fort Bend has been asked to investigate 15 percent of its schools and North Forest 20 percent.
That will be a tough deadline for HISD, which has yet to forward its investigation of Berry Elementary, dated November 2, to Lucio.
Lucio, who says he's never heard of Berry even though the investigation was reported by local media, is supposed to review all probes of TAAS-related improprieties within 30 days. After Lucio's review, the state can decide to punish educators even if school districts don't.
After the Berry investigation, HISD quietly got rid of Principal Kavulla (her contract was not renewed), a curious move considering the school was rated exemplary, and the allegations against Kavulla were supposedly "unfounded." Had the district -- which, Abbott says, is planning to submit the investigation report -- let the TEA review their findings, Kavulla might have faced further sanctions.
"It's always, 100 percent of the time, they protect the administrators," says union president Dansby. He adds that typically that protection extends to even those who are fired.
As for Kashmere High, Lucio says he has yet to receive a final report from the district. A year and a half has passed since the incident. "The last I heard from them, the investigation was still ongoing," Lucio says. Neither he nor his supervisor, Cruse, knew about the district's in-house readministration of the tests, or the precipitous drop in scores. "Normally, I should know about these things," Lucio says.
Yet Lucio is not losing any sleep over the matter. In fact, he affably offered excuses for HISD's tardiness, including the explanation that Kriendler has only had her job for "a couple of years."
"I'm pretty easygoing on this," Lucio says. "I'm not one of those people who goes around and actually uses strong-arm tactics."
The state's cavalier attitude stops just short of encouraging cheating. In fact, the way Shaw sees it, the Texas Education Agency has no more incentive than school districts to strong-arm cheaters. "There's certainly a suspicion in my mind that professionalism does not prevail in the state of Texas when it comes to testing and...making sure that everything is proper," Shaw says. "The agency is a political entity. They depend on their survival from the legislature and they're not about to shake anybody up."
As proof that the TEA really is on top of things, Smisko points to a major case last year where high-level officials in Austin ISD improved the district's ratings by manipulating data with what the agency called "laser-like precision." The fact that they were detected, she says, shows the system is working.
But the agency didn't catch on to the statistical sleight-of-hand until Austin schools inadvertently tipped them off, by calling to find out why their ratings had not yet been changed.
Even then, nothing happened to Austin ISD. "There is no punishment for the district itself other than the high level of embarrassment this will cause them," TEA spokeswoman Ratcliffe admitted to the Austin-American Statesman. "There is no legal sanction for the district." The case has prompted a state legislative proposal to make TAAS data-tampering a felony, and last week the Travis County Attorney announced that a grand jury would review evidence of possible criminal activity on the part of the district.
Smisko says Commissioner Moses has "aggressively pursued" complaints of cheating on the TAAS, yet the agency has revoked only one certificate for TAAS-related offenses, back in 1993. Since then, it has reprimanded or suspended certificates on five occasions.
"The bottom line is, we think educators are moral and ethical," says Smisko. "We believe they're doing the right thing and they're setting a standard for kids, and they're not in general, en masse, cheating on this test."
That may be true, but the incentive to cheat is hard to ignore. For every educator who comes in contact with the TAAS, high scores equate to money. The district gives employees two kinds of incentive pay. One is based on a campus accountability rating (determined primarily by TAAS scores), and divided among all school employees (except the principal), based on salary range, which means two teachers at the same school receive about the same amount, regardless of their classes' relative performance. The average payment last year, according to district figures, was $627.
Principals don't get incentive pay, but the majority in HISD earn $7,500 extra a year in exchange for having "performance contracts" that allow them to be terminated if they don't do their job. Clearly, that job includes improving TAAS scores. "A teacher doesn't have, economically, that much to lose if they have a bad TAAS year," Fallon points out. "The principals can find themselves unemployed."
Furthermore, incentives can work to the detriment of inner-city schools by driving teachers out. "Teachers have a very easy way to improve their TAAS scores," Fallon says. "Change schools."
The money trail leads all the way to the top. Superintendent of Schools Rod Paige has a vested interest in seeing the district's TAAS scores rise, as the amount of his annual bonus -- up to 25 grand -- is based solely on how well the district performs on TAAS. Texas Association of School Boards spokesperson Barbara Williams calls such contract provisions "rare." For two of the past three years, Paige has received all $25,000. In fact, Paige can get his bonus even if the district's TAAS scores go down, as long as the state's overall scores decline more.
Money aside, district and state both place a huge emphasis on the TAAS -- soaring scores are, after all, a great draw for businesses and good bragging rights for politicians. With Governor Bush's proposal to end social promotion by keeping children who don't pass the test from moving on to the next grade, a plan that gives children a genuine motive to perform well, the test's importance will only grow.
Critics of Bush's proposal complain the TAAS is a simple yardstick that measures a complex system. As long as the TAAS is the only yardstick, schools will continue to find ways to circumnavigate it. "[Cheating] is just one example of the systemic problem of trying to impact results beyond what the teaching of the children would reflect," says Shaw. Shaw points out that Texas' system is not unusual. In virtually every other state testing program, those who are measured and rewarded by the test are the same as those who administer it.
No matter how flawed the system, though, the public still perceives that high TAAS scores signify good schools. And for those who would rather send out the cheerleaders and marching bands than implement instructional methods that work for children, it's a simple matter to make sure a school has high marks to flaunt when the TAAS scores come back in the fall. "Those were the most sickening days, when we were talking about TAAS," remembers one teacher who left Kashmere Gardens after five years. "It would be like over and over; it would be shoved down your throat--'We're so wonderful! We're so great.' it was too obvious. When you're good, you're good, and you just let it lie."
Next week: Is the TAAS a wimp among tests?
E-mail Shaila Dewan at firstname.lastname@example.org.
Get the This Week's Top Stories Newsletter
Every week we collect the latest news, music and arts stories — along with film and food reviews and the best things to do this week — so that you’ll never miss Houston Press' biggest stories.
- J.J. Watt Is Damn Near Immortal, Wins Third NFL Defensive Player of the Year Award
Fri., Feb. 19, 6:00pm
Fri., Feb. 19, 6:30pm
Fri., Feb. 19, 8:00pm
Sat., Feb. 20, 1:00pm
- No, Houston Will Not Make a Lot of Money Hosting the Super Bowl
- Charged With a Crime? You Might Be Paying a Court Fee That Is Basically Un-Enforceable