Think tank wars

International Economy, The, Wntr, 2003

To the Editor:

We are somewhat underwhelmed by an "analysis" ("Think Tanks: Who's Hot and Who's Not," Fall 2002) by an Institute for International Economics scholar, Adam Posen, that purports to show IIE as the second most quoted economic think tank. At a minimum, the methodology is sloppy, with no explanation of its rationale. Why Foreign Affairs? Isn't including the International Herald Tribune with the New York Times and Washington Post double counting? Why Business Week but not Forbes or Fortune? Further, it's inexplicable that Cato's most quoted economist, Steve Moore, was excluded from the survey.

The International Economy is a fine magazine, so I'm surprised that you didn't consider the fact that the IIE came in 22nd out of 25 think tanks surveyed this year by FAIR in a far more extensive study. I believe it is odd that Brookings, Cato, and the American Enterprise Institute would be in the top four (along with the Heritage Foundation) in the FAIR study and that the only significant change was for IIE to move from 22nd to 2nd in its study. I would suggest you'd be on safer ground to limit these kinds of surveys to those undertaken by independent organizations.

EDWARD H. CRANE
President
Cato Institute
Washington, D.C.

Adam Posen responds:

We are glad to have the opportunity to respond to Mr. Crane's critical but unfounded assertions about the validity of our study of media citations of think tanks. This allows us both to clarify the methods we used in our research, and to correct those data errors and omissions which were brought to our attention since the data underlying our results were publicly posted (on the IIE website).

First, comparing the FAIR study of total media mentions of think tanks to our analysis misses the basic point--ours was a study of citations of economics think tanks and scholars of economic issues. Simply putting the word "Heritage" or "Brookings" without any specific scholars or subjects into a search engine, as the FAIR study does, lumps together every citation on every topic and does not serve the purpose that either we or The International Economy had in mind. We do not purport to study overall think tank mentions and explicitly acknowledge our limited focus in the article.

Second, any list of publications will in some sense be arbitrary. Why is it less arbitrary to take the Lexis/Nexis database's default list of "Major Newspapers and Transcripts" (which is what the FAIR study does) than to design a representative sample? Our sample of eleven publications consciously draws from a range of political views and selects those with wide readership and real economics coverage. Meanwhile, any search limited to Lexis/Nexis systematically undercounts certain publications, and double-counts others, which is why, as we explain in our methodology statement (also posted on the IIE website), we used both Lexis/Nexis and Dow Jones services for our searches. This is not a criticism of the FAIR methodology but underscores that search design is a choice along a tradeoff, wherein some conscious design may well be justified depending upon the study's goal.

Incidentally, Mr. Crane's misconception about the International Herald Tribune is another example of why we put such careful effort into our study design. Including citations in the International Herald Tribune and the Washington Post or the New York Times is not a double-count because they reach different subscriber bases and because there is no automatic reprint of Washington Post or New York Times articles in the International Herald Tribune. There are also economics articles that appear only in the International Herald Tribune written by their own staff.

Third, to test Mr. Crane's contention that our study misses a great deal by not including Forbes or Fortune, we performed an experiment. We searched for any mentions of the "Cato Institute," unconstrained by reference to specific economists, for the five-year period of our study. We found 48 citations. Were these added to Cato's total, and no credit given to any other think tank for any cites in Forbes and Fortune, Cato's total of 382 cites would still rank well behind AEI's third-place total of 662. This robustness of rankings suggests that we did use a representative and unbiased sample of publications in our study.

It is interesting to note that the main difference between our study's results and those of the FAIR study appears to be that IIE displaces Cato in the top ranks. That would seem to suggest that there is some benefit to Cato from the design of the FAIR study's sample of publications and/or its inclusion of non-economic issues that does not affect the relative positions of the other three institutions Mr. Crane mentions (Brookings, AEI, and Heritage), whose rank is essentially unchanged between the two studies when the focus narrows.

Fourth, the reason IIE is not in the top ranks in the FAIR study is self-evident--unlike Cato or Brookings or Heritage, we do not cover social, foreign policy, or other non-economic issues so of course we do worse in a general count of every press appearance by every think tank (including think tanks whose sole focus excludes economics), IIE's high ranking in our study is not the result of bias, but rather of the openly declared focus of the study.