poll tracks; Just How Do Those Numbers Add Up?

American Demographics, Nov, 1999 by G. Evans Witt

School children are studying away as the presidential candidates vie to demonstrate who has the most concern for educating the nation's offspring. Rhodes scholar Bill Bradley may have a leg up in this rhetorical contest, at least on George W. Bush, who likes to quip that he demonstrates how far a C student can go in America.

Are public schools up to the task of educating our children? Two recent polls asked in the most direct way possible what parents of public-school studen ts think about that.

The news from a Gallup poll for the Phi Delta Kappa education fraternity was dark indeed. When asked, "If you could send your oldest child to any public, private, or church-related school of your choice, with tuition paid for by the government," almost half of the parents of public-school students, 46 percent, said they would pick a different school. Only 51 percent said they would keep their child in the current public school.

But look at another poll on the same topic, this one by International Communications Research/ICR for a joint project of National Public Radio, the Kaiser Family Foundation, and the Kennedy School of Government at Harvard University. The poll asked: If the "government would pay all of the tuition for you to send your oldest child to a public school in another district, a religious or parochial school, or a private school, would you send your oldest child to the school he or she now attends or to a different school?"

In this poll, fully two-thirds of the public-school parents - 66 percent - said they would leave their child right where he or she is now. Only 29 percent said they would move their child to another school. Of that 29 percent, 6 percent of the parents said they would send their child to another public school. That adds up to almost three-quarters of public-school parents who would keep their child in the public schools.

So what are we to make of this difference? Is it that almost half of public-school parents would yank their children out of public school if they could? Or that nearly 75 percent would stick with the Board of Ed?

Both polls were professionally done, designed, and executed to high standards. They were conducted about the same time (May-June for Phi Delta Kappa and June-July for NPR). Each was based on a more-than-adequate sample. Could one of these be just a bad sample? Perhaps, but unlikely.

In addition, the wording of the questions was not dramatically different. The NPR query seems to emphasize a bit more the government's role in paying the tuition, but the distinction is relatively minor. Was it the order in which the questions were asked? After all, each poll asked question upon question about education. In the NPR survey, the public-school question was No. 37 on the list. Maybe the order swayed the results; no doubt there was some slight order effect.

The answer is probably the sum of many small factors, most of which can't be identified or duplicated precisely. But the point here is this: These results may not be as dramatically different as I have portrayed them. First, 51 percent in the Phi Delta Kappa poll said they would leave their children in their current schools. In the NPR poll, the finding was 66 percent - a difference of 15 percentage points.

But part of the problem is my use of "almost half" regarding those who said they'd move their kids, compared with "two-thirds" who said let them stay. Not only does the rhetorical shorthand of "almost" imply a greater contrast than is warranted, the comparison is actually misleading: Parents who answered in one direction to one question in the first poll compared with those who gave the opposite answer in the second poll.

What if the comparison had been "less than half" (46 percent) who said they'd move their child in the Phi Delta Kappa poll to "less than a third" (29 percent) who gave the same answer in the NPR survey? Suddenly, the difference looks less huge: still real at 17 percentage points, yes, but less monumental.

One final caution: These polls focused on parents of public-school children, who make up one-third or less of most general population samples. So if the margin of error for the entire sample is plus or minus 3 percentage points, the margin for results based on public-school parents is plus or minus 5 or 6 percentage points - for each poll. Suddenly, a 15 point or 17 point difference doesn't seem so big.

Take a little overblown rhetoric, add a good-size error margin, and mix in a sloppy comparison. Pretty soon you've misinterpreted the poll and misunderstood what the public is saying.

Not only does the poll have to be done correctly, but the analysis and reporting must also be done with equal care.