University rankings revisited

Issues in Science and Technology, Summer 1997 by Rosenzweig, Robert M

The Rise of American Research Universities, by Hugh Davis Graham and Nancy Diamond. Baltimore, Md.: Johns Hopkins University Press, 1997, 319 pp.

Robert M. Rosenzweig

To the list of life's very few certainties, Americans at least may confidently add a new category: ratings. We are an extremely competitive people, a fact reflected in our political and economic systems. Moreover, we are not content that every contest simply produce a winner and a loser; we yearn to know who among the winners is the best of the best. The proliferation of "best-of" lists is limited only by the imaginations of marketing specialists.

Until relatively recently, higher education was only marginally involved in the ratings game. Although intercollegiate athletics has been one of the main arenas of ratings madness, we hardly ever saw crazed university presidents, their faces painted, waving their index fingers and screaming, "We're number one," into TV cameras. Today, however, we have the functional equivalent of that finger waving in the reactions each year to the ups and downs of the institutional ratings published by U.S. News and World Report. It is surely one of the least savory developments in the recent history of higher education.

It is not, however, wholly unprecedented. In 1925, Raymond Hughes, president of Iowa State College, ranked 24 graduate programs in 38 universities. Others then quickly aggregated his data into institutional rankings. In 1957, Hayward Keniston undertook a systematic ranking of universities by asking department heads at 25 "leading" universities to rate the graduate departments. Since then, four national studies have provided fodder for institutional rankings. The two most recent, published in 1982 and 1995 by the National Research Council (NRC), are the most sophisticated and the most sensitive to the essential silliness of attempting to rank in order of quality entities as complex and diverse as universities. Alas, within days of the publication of the NRC studies, university public relations offices were cranking out analyses to the media demonstrating how well their institutions fared and/or why the study methodology failed to do justice to their splendid programs. Little, it seemed, had changed since 1925.

This history of ratings in higher education is recounted in useful detail in this excellent book by Hugh Davis Graham, professor of American history at Vanderbilt University, and Nancy Diamond, who has a Ph.D. in public policy from the University of Maryland, Baltimore County. Indeed, the book contains just about everything worth knowing about the attempts to rank U.S. research universities. Unfortunately, the authors point out, all previous studies relied heavily on rankings based on reputation. Reputation may be an increasingly perishable commodity in public life these days, but it is remarkably stable in academic life, and not always with justification. The well-known halo effect can mask declines in quality and dampen the perception of quality improvements. As Graham and Diamond write, "Reputational surveys, by capturing shared perceptions of institutions' rising and falling status in the academic pecking order, reinforce and prolong the reputations they survey."

New kids on the block

But the authors' interest in rankings is not the prurient one that puts U.S. News and World Report's annual higher education issue right up there with Sports Illustrated's swimsuit issue in newsstand sales. They have a serious point to make, and they make it convincingly: "The central argument of this book," they write, "is that new research universities did emerge from the competitive scramble of 1945 to challenge more successfully than has been realized the hegemony of traditional elites."

On one level, that conclusion seems so obvious as to be almost self-evident. The sheer number of universities that are now major actors in the research enterprise would seem to belie any notion of a system in which the rich get richer and the rest scramble for the leftovers. But in fact that notion has periodically dominated federal research policy and is the most commonly stated justification for the rise in academic pork-barrel spending. In the heat of politics and institutional aggrandizement, even what is obvious sometimes yields to what is advantageous, and the political advantage for some years now has been on the side of those who cry poor.

But there is more to the matter than simply the number of universities now in the research system as compared with some earlier period. The authors want to prove a further point: The rank order of universities as producers of research has changed, and a surprising number of newcomers have made it into the upper division of the big leagues.

As their instrument for demonstrating the change, Graham and Diamond have devised a set of indices that measures research productivity and eliminates the bias for sheer size that inevitably accompanies rankings that emphasize the volume of sponsored research. They focus instead on per capita publication output, especially publication in the leading peer-reviewed journals in the major disciplines. There are no perfect measures, including these, but it turns out that looking at universities over time using these measures is quite revealing and often surprising.