Correspondence

Education Next, Winter, 2001

Hurrah for Houston

Congratulations to Education Next for its trio of articles on the positive reforms in the Houston Independent School District, carried out in large part by Secretary of Education Rod Paige while he was the district's superintendent (see "Houston Takes Off," Feature, Fall 2001).

The article by Jane Hannaway and Shannon McKay of the Urban Institute showed how mandatory testing can serve as a lever to create structural change in an otherwise entrenched education bureaucracy. The piece by Marci Kanstoroom highlighted the benefits not only of testing students, but also of assessing their teachers.

Houston's reform program--systematically tying standardized tests to a back-to-basics curriculum and combining teacher training with recognition--does not involve punishing teachers whose students fare poorly. Instead, as Kanstoroom reported, the teachers receive the extra training they need to improve.

LARRY PARKER

Alexis de Tocqueville Institution Arlington, Virginia

Vouchers versus class size

The statistics reported in Dan Goldhaber's otherwise thoughtful article (see "Significant, but Not Decisive," Research, Summer 2001) inadvertently tilted the comparison between vouchers and class-size reduction in favor of vouchers. When comparable samples and measuring sticks are used, the improvement in test scores for black students from attending a small class based on the Tennessee STAR experiment is about 50 percent larger than the gain from switching to a private school based on the voucher experiments in New York City, Washington, D.C., and Dayton, Ohio.

In the D.C. voucher experiment, African-American students in grades 2 through 5 reportedly increased their scores by an average of 10 national percentile points in mathematics and 8.6 points in reading after two years of private schooling. These percentiles are based on the national distribution of scores on the Iowa Test of Basic Skills. Goldhaber scaled these gains by dividing them by the standard deviation of percentile ranks for African-American students in the control group and concluded, "This represents a gain of about 0.5 standard deviation relative to African-American students whose applications for vouchers were unsuccessful." He compared this figure with Jeremy Finn and Charles Achilles's finding that attending a smaller class in the Tennessee STAR experiment raised reading scores for black 2nd graders by one-third of a standard deviation.

Finn and Achilles, however, measured test scores using a different metric--namely, "scale scores" on the Stanford Achievement Test. Because the scale scores follow a bell-shaped distribution, while percentile ranks are uniformly distributed between 0 and 100, a one-standard-deviation increase in scores does nor imply the same improvement in achievement in the two measures. Furthermore, these effect sizes are not comparable because the standard deviation used to scale the voucher results is from a much less diverse sample: low-income, inner-city students who participated in the experiment.

Another problem is that the effect sizes Goldhaber took from the Washington, D.C., voucher experiment were adjusted to account for imperfect compliance--the fact that not everyone offered a voucher attended private school, and some of those who weren't offered a voucher nevertheless attended private school. If the same approach is applied to the STAR sample to adjust for the fact that some students did not enroll in the class they were assigned to--and a comparable sample of low-income black students is used--the gains in test scores after two years of attending a small class (average of 16 students) as opposed to a regular-size class (average of 23 students) is 9.1 national percentile ranks in reading and 9.8 ranks in math.

Note also that Goldhaber compared the STAR results with voucher results for just Washington, D.C., where the gains were higher than in Dayton or New York, Across all three cities, the average effect of switching from a public to a private school for black students was 6.3 percentile ranks in both math and reading. This would seem a more appropriate comparison.

Then there is the question of how much of the students' gains in private schools are attributable to the fact that private schools had smaller class sizes in all three cities. In Washington, for example, the average class size attended by students who switched to private school was 18, compared with 22 for those who remained in public school. The gain attributed to private schooling may be due to smaller classes.

Similar issues arise in William Howell et al.'s article reporting results from the voucher studies (see "Vouchers in New York, Dayton, and D.C.," Research, Summer 2001). They scale the gain in black students' scores by the standard deviation of test scores computed for a select sample of students, and observe that the gain in their scores due to attending private school is "roughly one-third of the test-score gap between blacks and whites nationwide." The nationwide gap, however, is presumably scaled by the larger nationwide standard deviation, The standard deviation Howell et al. used to scale gains was around 19, while the standard deviation of national percentile scores is necessarily 28.9, because percentile ranks follow a uniform distribution. Using the national standard deviation to scale all scores, the effect of attending a private school on black students is only one-fifth to one-quarter as large as the blackwhite gap.