Measuring Canadian business school research output and impact

Canadian Journal of Administrative Sciences, Jun 2002 by Erhan Erkut

It is also possible to rank schools using citation credits or paper credits per capita, by dividing the credits by the number of faculty members for each school. When faculty sizes are taken into account, McMaster moves up (to first spot in paper credits and second spot in citation credits), Toronto and HEC move down, and Carleton enters the ranks of the top schools. Figure 6 displays the top 10 business schools according to citation credits per capita.

In this paper, we calculate citation credits by dividing the citation count of each paper by the number of coauthors. While this is a popular method in scientometrics, it is not universally accepted. The comparative analysis can also be carried out using the raw citation counts (in effect double-counting citation credits with multiple Canadian coauthors). If we do this, much of the rankings remain the same. (The following pairs switch spots: Toronto and HEC, York and Ivey, McGill and Queen's.) For completeness, we provide this additional information in Table 5.

Trend Analysis

The results presented in the previous section provide a historical snapshot of research output and impact by Canadian business schools. One may also be interested in what trends may exist in these measures, to see what the future may hold for Canadian business schools. If we assume that the faculty memberships will remain unchanged in the short run, a limited trend analysis is possible. Table 6 contains the data necessary for this analysis.

There are many ways of estimating trends. Beamish (2000) uses the difference in the paper credits between the second and the first half of the decade. Another obvious alternative is to fit a line to the 10-year paper-credit data and check the slope. Most of the slopes for our data are close to zero, implying that there are no obvious trends. In fact, the only one that is significantly different from zero, at a 95% confidence level, is McGill's positive trend of 0.85 (P-value = 0.014). If we focus on the last few years, we notice that the outputs of York, McGill, and HEC all peaked in 1996 and then went down gradually over the next three years, while the output at Ivey went up over the same time period. If we compare the average output in the last two or three years to their 10-year averages, we see that the most recent outputs of Ivey and McGill are considerably higher than their averages, while the outputs of McMaster and Queen's are considerably lower than their averages. Toronto and Alberta also have below-average outputs in the last two years. Our limited trend analysis suggests that there may be reason for concern for all of the top schools except UBC, Ivey, and perhaps McGill.

It would be useful to have some feel for the trend in citation credits as well. However, it is not possible to use a time-series method to compute trends or forecasts using citation credits since we do not have annual datawe only know the number of citations as of May 2001. It is possible, though, to construct a relatively simple model to predict the number of new citation credits for the current set of papers in the coming year. The model we developed (detailed in Appendix B) takes into account three important factors for each paper: the current citation count, the year of publication, and the publishing journal. Using our model, we forecast the additional citations to be credited to the set of papers in our database and allocated them to schools accordingly. For the top nine schools, the average of future citations, as a percentage of current citations, is 36%. (This may appear high, but note that we are keeping every paper in the database and "aging" each of them by one year.) The school that benefits the most from the citation count increases is UBC (43%). HEC, Alberta, Queen's, Ivey, and Toronto experience increases of between 37% and 39%. The universities benefiting the least are McMaster (33%), McGill (31%), and York (27%).