Cooking the questions: Phi Delta Kappa's annual poll is regarded as the definitive measure of where Americans stand on education issues. But are its surveys biased and its reporting self-interested?

Education Next, Spring, 2002 by Terry M. Moe

The first two essentially set up a dichotomy between vouchers and the public schools. The second of them asks, "Which one of these two plans would you prefer--improving and strengthening the existing public schools, or providing vouchers for parents to use in selecting and paying for private and/or church-related schools?" PDK is thus clearly suggesting to respondents that people who support public education--as most Americans do--cannot at the same time support vouchers. From a framing standpoint, this is a killer. It is also factually incorrect. Most activists in the voucher movement are dedicated to improving the public schools, and they see vouchers as a powerful means of effecting improvement through greater choice and competition.

The next three new items are also problematic. Two focus attention on a long list of public-school ideals. The third contrasts parental choice with other"possibiliries"--like rigorous academic standards and competent teachers--again giving the impression that they are alternatives to vouchers rather than (as is in fact the case) entirely complementary.

There is little doubt, in my view, that the introduction of these five new items just prior to the usual voucher items produced a more negative framing of the voucher issue and encouraged the lower support scores that became PDK's findings in 2000, The same sort of thing happened in 2001, except in that year PDK's researchers changed the survey again. This time, they eliminated three of the five lead-in items, and included just the two killer items: the ones that portray vouchers as antithetical to public education (see Table 1). Whether this lead-in is more or less negative in its framing than the 2000 lead-in is unclear. I suspect it is more negative, because it is so simple and forceful and avoids all the distractions of the other three items, In either event, it is surely more negative than the original framing from 1999 and before, and there can be no surprise that it again led to lower support scores.

The most reasonable conclusion, therefore, is that the "significant decline" in voucher support--loudly proclaimed by PDK and reported by the media as fact--is an artificial phenomenon of PDK's own making. The important changes didn't occur in public opinion. They occurred in the design of PDK's survey--a factor that, needless to say, is under the conscious control of PDK's own researchers.

Conclusion

I do not want to believe that PDK is using its survey to further its own political agenda. But what is the alternative? That PDK's researchers have simply been careless in their design decisions? That these decisions have by sheer accident led to lower support scores for vouchers? That the most biased of these scores have unwittingly been urged on the media as good-faith evidence of American public opinion? This is a lot to swallow, It is much more reasonable to believe that PDK's researchers are competent at their jobs and that they have not been making one mistake after another.