School Voucher Programs: What the Research Says About Parental School Choice

Brigham Young University Law Review, 2008 by Wolf, Patrick J

Moreover, cross-sectional studies of private schooling or school choice typically rely heavily upon participation in federal government aid programs as variables to "control" for selection bias in private-public school comparisons.38 The rates of school-level participation in such programs are much higher in the public sector than in the private sector.39 For example, student disability status is signified in public schools by a student having an Individualized Education Plan (IEP). Students with disabilities who switch to private schools remain disabled but surrender the IEP label.40 While the federal government free and reduced price lunch program is offered to students in all public schools, school-level participation in the federal lunch program is discretionary for private schools. Many private schools decline to participate in the federal lunch program because of the extra administrative burden involved.41 For these reasons, a student with the exact same low income and disability is much more likely to be a participant in the lunch program and have an IEP if he or she attends a public school than if he or she attends a private one.42 As a result, modeling selection by including a control variable for participation in the federal lunch program or having an IEP has the practical effect of controlling for the negative effects of low income and disability on test scores among public school students but not among private school students. The predictable effect of such a flawed analytic approach is that the estimate of the "private schooling effect" becomes a negatively biased combination of the true private schooling effect minus the effect of being low income and disabled.43

The third major flaw in cross-sectional analyses of voucher effects is that they are static in that they rely exclusively upon measures of variables at a single point in time. Such studies do not and cannot examine change or growth that is a result of private schooling or school vouchers because their data consists of a single snap-shot of students.44 Based on this shortcoming in observational studies, the Charter School Achievement Consensus Panel, a national panel of research experts assembled to evaluate various methods of evaluating school choice interventions such as charter schooling, concluded, "studies using one-year snapshots of achievement cannot have high internal validity, no matter how large a database they draw from or how carefully the analysis is done."45 Robert Boruch sums up the basic weakness of quasi-experimental analyses of cross-sectional data thusly:

Analyses of data from passive surveys or nonrandomized evaluations or quasi-experiments cannot . . . ensure unbiased estimates of the intervention's relative effect. We cannot ensure unbiased estimates, in the narrow sense of a fair statistical comparison, even when the surveys are conducted well, administrative records are accurate, and analyses of quasi-experimental data are based on thoughtful causal (logic) and econometric models. The risk of misspeciBed models, including unobserved differences among groups (the omitted variables problem), is often high.46