good reviewer, The

Academe, Nov/Dec 1999 by Chilton, Stephen

Peer reviewers get a little formal guidance. Following a few simple rules can make the process more useful to authors and editors alike.

PEER REVIEWERS HAVE A LOT OF POWER. THEY help editors of academic publications decide what gets published and thus indirectly help deans decide who gets tenured. But editors and deans are not their only audience; peer reviewers also influence authors. Reviewers can offer feedback to authors, enabling them to improve their work, and they can also give authors recognition: a sense that the enterprise in which they are engaged has merit, regardless of the virtues of their article. Such recognition is critical for junior authors, but even senior authors value it. And our disciplines can only benefit from an atmosphere of mutual appreciation of one another's efforts.

Yet despite its importance, peer review tends to be uneven in quality. Probably most authors have had the frustration of getting mere summary comments from a reviewer who did not seriously address a manuscript's argument-or even understand it. These problems arise because peer review is hard to do well. Here are a few of the difficulties that can disrupt the process:

Academic competitiveness. Most peer review is done by people with some stake in the outcome. Keeping the author's identity from a reviewer cannot disguise the intellectual impact a work will have on an academic field. The reviewer may recommend against publication of a manuscript that he or she finds threatening.

Lack of training and recognition. I received no training to review my peers, and I doubt many people do. Graduate school teaches us to find and criticize logical flaws, but peer review requires far more than that, as the guidelines below should show. Moreover, faculty members receive little professional recognition for peer reviewing, regardless of how well they do it.

Difficulty of dealing with original work. The more original the work, the more difficult it is to evaluate. It may use new methods, test the dominant paradigm, or even employ standards of a new paradigm. Those who review such works always face the problem of how to judge the new by the standards of the old.

In the hope that we can all thread our way through these problems, I offer below some guidelines and suggestions I've found useful in my own reviewing. Certainly, I would like my own work to be reviewed along these lines.

Basic Structure of the Task

THE PEER REVIEWER IS RESPONSIBLE TO BOTH THE EDITOR and the author simultaneously. Some reviewers feel they must defend authors against censorious editors or editors against inferior manuscripts. But the task of reviewing, property conceived, involves no such conflict. Both the editor and the author benefit from a straightforward appraisal of the manuscript.

It is useful to start by reading a manuscript's introduction and conclusion, as well as the abstract (if it has one), since your task includes judging the work's logical flow. But do read the entire manuscript. That should be obvious, but at least one journal editor has turned in desperation to excising all abstracts, because too many reviewers read only the abstracts.

Draft manuscripts are more difficult to read than published works, so peer reviewers should expect to work harder than reviewers of published works. The author is, however, responsible for providing a readable manuscript. If the author has not done so, I feel little obligation to finish reading it. I write a short, polite, but definite statement about the writing problems, suggest that the author seek the help of a freelance copy editor, and pass along whatever useful comments I can on the work's argument.

In addition, I would suggest that you confine your comments to the manuscript; don't review the author, and don't be afraid to say no. Avoid second-guessing what the journal needs or the editor wants. The editor gets to decide what to publish; a good editor wants your professional judgment.

Take sufficient time to review the work thoroughly, but return your comments no later than three weeks from receipt of the manuscript. If you cannot commit to doing so when you receive it, either return the manuscript immediately or ask for an extended deadline.

ADMIT (IF ONLY TO YOURSELF) THAT YOUR REVIEW GIVES your judgments, not God's. Though your editor needs you to explain your judgments clearly, you can still phrase them in a way that acknowledges their fallibility. The Golden Rule is appropriate here, not in a Pollyannaish sense but rather in the sense of asking yourself, "Given that this manuscript has problems X, Y, and Z, how would I like to be informed of these problems if the work were mine?"

The anonymity of reviews seems to encourage abusive behavior. I value anonymity as an aid to detachment. Still, I start each review by typing in my name, even though I remove it from the final version. I know logically that the author won't see my name, but its presence on my drafts keeps me psychologically attuned to being respectful.

State appreciatively what is good about the manuscript. Every work has something good about it: new data, a new approach, interesting insights, a breadth of perspective, grace of expression, and so on. In particular, recognize and state appreciatively the importance of the problem the author is trying to address. Even if its significance is unclear to you (and possibly even to the author), authors always strive to speak to something important. Encourage that.