The problem with polling

Public Interest, Summer, 2002 by Robert Weissberg

WOULDN'T it be wonderful to live life unencumbered by the laws of economics? Consumers could set their own prices, both guns and butter would be plentiful, and risk would have been banished like some ancient disease. This hope is obviously utopian, and history shows that disaster awaits idealists and demagogues who insist otherwise. Though all of us have our favorite economic theories and castigate our opponents as charlatans, some economic realities are indisputable. Imagine if an office seeker suggested that Congress amend the law of supply and demand to bring prosperity. Such a proposal would be laughable and surely not admissible in public discourse.

Or so it would seem. Yet an economics-free utopia does exist, and, most critically, it is politically consequential. This is the world of public opinion polling. When the public's desires for government social-welfare spending are assessed, pollsters in effect grant respondents diplomatic immunity against the laws of economics. The problem is not "loaded" questions concocted by partisans to secure some immediate political advantage. Every item in public opinion polls is fully certified by the survey experts who write the official textbooks and attend the professional gatherings. So secure is the grip of the polling establishment that even the economically literate occasionally take their polls seriously.

While wording details and the menu of alternatives vary, the poll's paradigmatic question is generally "Do you want the government (usually federal government or 'Washington') to spend more (or less or the same) to solve the problem of X?" Decades of applying this polling formula have confirmed that Americans are fervently attached to the social-welfare state. This is true both at the general level, for example, for spending more for education and health care, as well as for narrower, more controversial issues such as increased funding for AIDS research. Even Republican electoral victories cannot undermine this consensus regarding the public's generosity, though, to be sure, numbers do fluctuate with events.

The advantage of this polling method is the technical convenience for both pollsters and respondents. Terse repetitive questionnaire items coupled with fill-in-the-blank policy choices permit quick "advice" on dozens of issues via inexpensive telephone interviews, a major commercial benefit for sponsors of polls. Questionnaire writers need not master intricate policy issues when asking about "more money to improve health care." Unfortunately, however, ease of questionnaire execution and the extraction of economically sound policy counsel are inversely related. Polling's inattention to even the most elementary economic realities renders the collected data nearly worthless, yet unfortunately, politically significant.

A free lunch?

The most obvious economic reality ignored by today's pollsters concerns financial costs and the revenue necessary to fund government programs. Admittedly, pricing contentious entitlements can be arduous, and presenting this information to the unsophisticated is time-consuming. But these impediments do not justify discarding marketplace factors altogether. Even Soviet command economies offered at least some pricing. Imagine quizzing potential consumers about a new product without mentioning costs. Nobody would accept the results as useful. Yet this is what happens in surveys about government entitlements all the time.

With scant exception, conventional survey questions avoid any mention of the costs to the taxpayer. For example, a 1989 Gallup Poll (typical of those on this subject) asked whether school class size should be reduced to 15 students, an idea that, as expected, drew great support. Glass size reduction is an extremely expensive venture, however, easily costing hundreds of billions of dollars nationally--a fact that the poll never broached. Sometimes expenditure levels are merely implied, with respondents free to imagine their own figures. The National Opinion Research Center, the University of Chicago's acclaimed polling organization, has long favored the "too much money," "too little," or "about the right amount" phraseology to assess desires for new entitlements. An even simpler (and more popular) format is just asking if the interviewee would be willing to spend "more," "less," or "the same" for the enumerated programs.

Excluding tangible costs from polls opens the door to immense mischief. Respondents may differ greatly in the amounts they have in mind when saying "more," no small issue since most political disputes revolve around how much to expand government programs. Moreover, these silently imagined figures might not be sufficient to accomplish the policy's stated objective. My own research using a national random telephone sample of 550 respondents into federal aid to reduce class size departed from this "more/less/same" convention. Respondents were merely asked, "How much would you be willing to pay for reducing class size to 15?" Confronted with even the idea of a bill calibrated in actual dollars, generosity wilted. Thirty-one percent of those favoring this approach refused to pay anything to bring it about, and only about a quarter agreed to pay the program's official cost (and even this figure is deceptive since the actual cost of the initiative is far more than its official price tag). Polls that substitute "mor e" or "less" for accurate estimations of costs are the equivalent of supermarkets using terms like "less than you think" or "great at twice the price." Imagine checkout counter howls when such vacuousness is translated into hard numbers. No business of that sort could possibly survive.