Student Perceptions of High Course Workloads are Not Associated with Poor Student Evaluations of Instructor Performance

Journal of Engineering Education, Jan 2007 by Dee, Kay C

Pearson correlation coefficients and Spearman's rho correlation coefficients were first calculated from the Tulane data. Statistical outliers, or courses with at least one course-averaged item score more than three standard deviations away from the overall mean score for that item, were identified and removed from the data set. Pearson and Spearman's correlation coefficients were then recalculated for this data set. The Pearson correlation coefficients were compared with previously-reported Pearson correlation coefficients obtained from the same original data set after outliers were removed, a natural log transform was applied to the scores, and z scores were subsequendy calculated from the log-transformed data [8]. The transforms and z scores reduced skew and equalized variances, making the data fit better with the assumptions inherent in the use of the parametric Pearson correlation (see reference [8] for detailed data descriptors, discussion of model adequacy checking and data transformations, and a complete set of inter-item correlations from the Tulane data). The (previously-reported) Pearson correlations calculated after multiple data transformations and the Pearson correlations calculated (in the present study) after the removal of statistical outliers were only slightly different (mean change of 0.017 across all items, maximum change of 0.04 on any one item) from Pearson correlations calculated from the original data set with no data transformation or outlier removal. Spearman's rho correlation coefficients calculated after removing outliers were only slighdy different (mean change of 0.007 across all items, maximum change of 0.02 on any one item) from Spearman's coefficients calculated without removing outliers. Subsequent data analyses utilized solely Spearman's rho correlation coefficients and original data sets with no alterations or transformations.

Spearman's correlation coefficients were calculated from the Rose-Hulman data for all courses, for engineering courses only, for mathematics, science, and humanities courses only, for mathematics and science courses only, and for humanities courses only. Selected correlation coefficients were quantitatively compared using Fisher's Z statistic [10], a method of testing whether two population correlation coefficients are equal. Linear regressions were conducted on selected items from the Rose-Hulman data, and the slope and intercept values from the regression lines were statistically compared using t-tests [9]. Data from the Rose-Hulman engineering courses were sorted into quartiles according to the numerical ratings of the overall instructor performance. In other words, courses were sorted in descending order of overall instructor performance scores; courses in the top 25 percent of overall instructor performance scores were considered "highest quartile" courses and courses in the bottom 25 percent of overall instructor performance scores were considered "lowest quartile" courses. Mean scores on each evaluation item from courses in the highest and lowest quartiles (e.g., ostensibly viewed by students as the "best-taught" and "worst-taught" courses) were then compared using the Mann-Whitney test, a nonparametric way of determining whether two independent samples are from the same population [9].