Newspaper accuracy: A new approach

Newspaper Research Journal, Winter 1994 by Kennedy, George

Mitchell Charnley concluded his pioneering 1936 study of newspaper accuracy with the hope that future research would provide "a reliable body of data concerning newspaper dependability...," which eventually would mean that "newspapers--and teachers of journalism--could concentrate intelligently on bolstering the weak points in reportorial and news writing techniques.(1) During the more than 50 years since Charnley wrote, both the newspaper industry and journalism scholars have concentrated on building the body of data and on bolstering accuracy.

However, neither effort has reached satisfactory conclusions. The extensive research has revealed as many complications as solutions. And the industry has satisfied neither itself nor its readers that its reports are adequately accurate. Indeed, a national survey, sponsored by the American Society of Newspaper Editors a half-century after Charnley, found perceived inaccuracy at a level strikingly similar to that he found. The authors of that study, which was intended to assess the broader concept of credibility, offered as their first recommendation for improving credibility: "Be fair, unbiased, accurate, complete, factual, professional, aggressive and compassionate.(2) Similar injunctions to be accurate are found in all the profession's codes of ethics.

From the beginning, scholars and journalists have agreed that the primary tool for measuring accuracy should be the assessments by sources themselves. Thus, in his first study Charnley sent selected articles to people identified in the articles as sources of information and asked those people to point out and describe any errors in the published work. Along with the identification of inaccuracy, the study also identified a problem that continues to plague scholars and practitioners. The most common type of error cited by the sources was "errors in meaning.(3) The problem was, and is, that meaning lies largely in the mind of the beholder. Context, purpose and perspective all affect meaning. Agreement about meaning is far less likely than is agreement on such objective content as spelling of names, titles or numbers. (Errors in names and titles were the second- and third-most common types Charnley found.)

The significance of this distinction between objective and subjective errors was well illustrated in William Tillinghast's effort to explain the sources of mistakes. Tillinghast followed generally the well-established procedure of asking news sources to assess the accuracy of newspaper articles. Then he went back to the reporters who had gathered the information and written the articles. He found that reporters were willing to concede mistakes in about half the cases identified by sources as objective errors. However, when sources claimed subjective errors--such as misleading context, misplaced emphasis or inadequate explanation--reporters agreed in only about 5 percent of the cases.(4)

Newspaper efforts to concentrate intelligently on improving accuracy have relied for years on a tool whose own reliability is, in fact, dubious. That tool is an adaptation of the original Charnley methodology, now usually called the accuracy check. Typically, published articles are clipped from the paper and mailed to one or more of the information sources identified within the articles. The clippings are accompanied by a letter from an editor and, usually, a form on which the source is invited to indicate any errors the source finds. The forms also usually ask about the fairness and completeness of the coverage and solicit open-ended comments. These checks are popular with editors, who regard them as serving multiple purposes. At a national conference of newspaper ombudsmen, for example, accuracy checks were described as "a good opportunity to remind the public and reporters and editors that credibility and fairness are the cornerstones of our payer.(5)

However, just as definitional problems undermine the validity of scholarly accuracy checks so do methodological problems call into question the reliability of accuracy checks administered by newspapers. Gilbert Cranberg conducted an accuracy check of the Des Moines Register and compared his results to those obtained in a check by the Register itself. The difference in result was striking. The Register received complaints of inaccuracy from only 14 percent of sources. Cranberg got such complaints from 63 percent.(6)

In his report to editors, Cranberg concluded that the discrepancies are so great that "editors ought to embark on accuracy-checking with caution... . If accuracy-checking is worth doing, it's worth doing: in a way that assures editors and staffers get all the bad--as well as the good--news about accuracy."

The most elaborate effort to date t achieve that assurance was conducted by another journalist/scholar, Philip Meyer. Meyer sought to overcome the problems identified by Tillinghast and Cranberg. His goal was to create a feasible management tool for newspapers to use in measuring their own accuracy. His methodology is described in detail in the Newspaper Research Journal.(7) He attempted to bridge the gap between source and reporter by introducing a third-party assessor to determine, after weighing source complaints and reporter responses, which of the complaints had merit. He concluded that such an approach, in any one of several variations, could be workable.