Improving a School's U.S. News and World Report Ranking

Journal of Engineering Education, Jul 2004 by Tsakalis, Konstantinos S, Palais, Joseph C

III. ANALYSIS OF THE U.S. NEWS AND WORLD REPORT DATA AND RANKING SYSTEM

Regardless of the details of computation of the U.S. News and World Report scores, we can treat the entire process as an unknown system and identify its behavior from the given "input-output" data. The usual caveat with such an approach is that the extrapolation properties of the model depend greatly on the assumed model structure and the range of the data limits the model validity. For this reason, it is instructive to review the data before performing any data fits. The scores of the top 50 schools are plotted in Figure 1(a). The school ranking as a function of the total score exhibits smaller slopes for lower-ranked schools. In the range 40-50 the scores are within three points.

An examination of the correlation of the total scores and the individual sub-measures shows that some sub-measures have a relatively strong correlation, while others are either uncorrelated or weakly correlated with the total score. More specifically (as noted in Table 1) the peer and recruiter scores, the number of Ph.D. graduates, and the expenditures per faculty member show the strongest correlation with the total score (their correlation coefficients are in the 0.8-0.9 range). On the other hand, the selectivityrelated sub-measures show the least correlation with the total score. The other sub-measures exhibit weak correlations (coefficients 0.5-0.6). In addition to the U.S. News and World Report submeasures, we also calculated the correlation between faculty engaged in research and total score and number of full-time faculty and total score. Although relatively weak (coefficient 0.6), these correlations are important because it connects an apparently independent variable (school or research program size) to the total score.

It should be emphasized at this point that since the various submeasures are not independent, such statistical correlations should be interpreted very carefully. Their validity is restricted to the region covered by the data. Some important correlations among sub-measures are given in Table 2. With this in mind, we can attempt to fit the data by a linear model as in Figure 1(b). The structure of this model is a linear combination of the eleven variables and a constant offset term. The latter is motivated by a Taylor linearization of the logarithmic transformation, used in the computation of the final score. Despite this nonlinear transformation, it turns out that the linear model fits the data reasonably well with an error standard deviation of 0.94, (score standard deviation 16.4) and no apparent nonlinear effects. The model parameters are:

Because of low correlation between GRE scores and the total score and selectivity and the total score, their corresponding coefficients are not reliable. The rest can be interpreted as sensitivities of the total score to small variations in the data. For example, the research expenditures parameter suggests that a one-point increase of the total score can be achieved by a $10M increase (the inverse of the coefficient 0.0984) in expenditures. Such an interpretation, however, hinges on the assumption that the individual sub-measures can be manipulated independently. This assumption is rather unrealistic since an increase in expenditures is intuitively expected to affect several other sub-measures as well. Consequently, this linear model has limited value in establishing a direction for improving a school's ranking. However, it may be useful as a predictive model (i.e., for predicting a school's total score based on its projected individual scores).