Measuring Canadian business school research output and impact

Canadian Journal of Administrative Sciences, Jun 2002 by Erhan Erkut

Some authors are not consistent with the name they use in their publications. By way of an example, Professor Robert Adam Simpson may use any one of the following names on a given article: R.A. Simpson, R. Simpson, B. (Bob) Simpson, or B.A. Simpson. Likewise, some individuals may change their names (for example, due to marriage). The common wisdom in academia is to use the same name consistently in all publications, but not all academics adhere to this practice. While we do not believe this is a significant source of error, we believe that we may have missed some publications for this reason.

Citing authors may misquote a reference by using an incorrect spelling of the name, an incorrect (or missing) volume or page number, or an incorrect publication year In our study, we found that errors by citing authors were quite common. Such errors have a direct impact on a citation study since an incorrect citation cannot be linked to the source publication in the ISI database. To make matters worse, typos in reference lists have a tendency to propagate due to referencing based on someone else's (incorrect) reference list. In addition to incorrect references, citations to preprints of an article (working paper or research report), as well as citations to articles "inpress," are not included in citation counts. We believe the undercounting, due to errors such as these, is nontrivial. (In fact we are aware of a few instances where the erroneous citations outnumber the correct citations.) Unfortunately, we know of no easy way to correct these problems. We emphasize that, however imperfect, the same measure is used across all individuals and faculties so none are at a distinct disadvantage from this type of error.

Authors with common names may have been credited with citations to papers they have not written, or some of their papers may have been omitted. For example, a search for papers by Smith R* results in 7,940 references in a variety of disciplines. In some cases, a visit to the individual's web site allows one to determine which of the listed papers are authored by the individual under consideration. However, in many cases, those conducting the search had to use their judgment to include or exclude papers, and undoubtedly some mistakes were made.

In an attempt to purge our database of as many errors as possible, we undertook the following steps before compiling our final statistics:

* We double-checked the paper and citation data for the top nine schools discussed in this paper.

* We asked for assistance from all Canadian business school deans on two separate occasions. A total of 17 schools responded (see the acknowledgments on the first page), which helped to improve the database.

* We made the data public via our web site, announced it at the ASAC Conference in June 2001, and invited academics to submit corrections. The web site registered 1625 hits by April 2002. About 20 academics sent us messages asking for corrections, and all suggestions received by April 2002 were considered.