By Jeff Balke
By Aaron Reiss
By Angelica Leicht
By Dianna Wray
By Aaron Reiss
By Camilo Smith
By Craig Malisow
By Jeff Balke
But what of subjects that the TAAS test doesn't cover? In science, which is only tested on the eighth-grade TAAS and is not included in state accountability ratings, the NAEP scores plummet; Texas ranks 28th of 30 and its Hispanic students 21st of the 24 states who gave a representative Hispanic sample, lending some credence to critics who say subjects tested by TAAS are emphasized to the exclusion of others.
NAEP reading and math figures lose some of their luster under close examination. First of all, the test sample is small, about 2,000 to 3,000 students per test. Second, national scores over time haven't improved all that much. Between 1992 and 1996, Texas fourth-graders jumped 11 points in math, on a 500-point scale. That's about the equivalent of two points on a 100-point scale. The gain bumped Texas from 19th to sixth in the country.
In fourth-grade reading, Texas's average score on the NAEP actually dropped one point, from 213 to 212 points out of 500, between 1992 and 1994, the only two years that test was given.
"You have to break it down," says the Texas Business and Education Coalition's Stevens, explaining that the largest gains on the NAEP can be seen by ethnic group. "The overall result is lower because Texas is getting much more diverse." In other words, since blacks and Hispanics score lower on average than whites and Asians, a growing population of blacks and Hispanics means a lower overall average, even if the scores for those particular ethnic groups go up. Stevens's contention carries in math, but not in reading: Black and Hispanic scores in reading went down between 1992 and 1994, a period which covers only the very beginning of the accountability system.
As for the commercial standardized test Texas gives, there's not much comparative data, since the Legislature has frequently changed its mind about which test should be given. For 1995 and 1996, the state sampled 12,500 students in each grade (three through eight and ten) and used the seventh edition of the Metropolitan Achievement Test (MAT7). In 2000, the state will give the MAT7 and will continue to do so every three years, unless the Legislature changes the law again.
In reading, Texas's average percentile ranking on the MAT7 lingers slightly below the national average, and between 1995 and 1996 most grades gained between one and two percentage points in reading.
In math, the news was not as good: Although scores hovered above the national average, most grades declined between one and four percentage points on the MAT7 percentile ranking, yet TAAS passing rates went up about five points.
So on the NAEP, math is up and reading is down. On the MAT7, reading is up and math is down. On the TAAS, both math and reading are up.
Is that proof that the TAAS is valid? The TEA's Pat Porter says the TAAS is more sensitive to instruction than norm-referenced tests and children are more highly motivated to perform well on it. Gains on the TAAS, she says, "are validated by our gains on the NAEP, and our students are at the national average on a norm-referenced test."
Shaw says the fact that the TAAS is calibrated to instruction makes it easier to beat. "Every time you give a test it loses part of its validity, because what you're trying to do is see what the curriculum is doing in this area without influence of the test. And everything a teacher learns about the test, they teach more directly to the content ... So we don't know from the TAAS itself how effective our curriculum is."
To Shaw, celebrating the state's gains on the TAAS is a way of sidestepping the point: "Why is anybody missing any of these questions?" Shaw asks. "We've gotten very little bang for our buck on TAAS improvement. We ought to be hanging our head in shame."
Non-TAAS indicators of educational health aren't so hot. While eighth-grade passing rates on TAAS went from 68 to 83 percent from 1996 to 1998, a gain of 15 points, the percentage of students passing the end-of-course algebra test (usually taken in eighth or ninth grade) increased only 11 points, to a still-dismal 39 percent passing, and that on a test that combines prealgebra with algebra.
And while the state says the dropout rate is 9.1 percent, the so-called "on-time graduation rate" tells a different story: About 60 percent of the number of students who enroll in ninth grade get diplomas four years later, and that number declined slightly from 1995 to 1997. For blacks and Hispanics, the dropout figures are worse: Less than half get diplomas after four years. Because so many students -- 21 percent -- disappear after ninth grade, they don't even take the exit-level TAAS.
The one organization dedicated to helping states evaluate their assessments hasn't attracted much attention from Texas. Achieve Inc., founded by state governors after the 1996 National Education Summit, was set up to provide comparability and continuity among states in the growing national trend toward high-stakes accountability systems and testing. The organization can conduct a "rigorous review" of any state's accountability system and helps states compare their tests to others nationally and internationally.
But Achieve's Director of Standards and Assessment Matthew Gandal says that, although there's "been some interest" from Texas, the state hasn't sent anyone to either of Achieve's annual meetings (out of 25 states invited, 21 are participating), nor has it asked for a one-on-one review.