Relations among student effort, perceived class difficulty appropriateness, and student evaluations of teaching: is it possible to "buy" better evaluations through lenient grading?

College Student Journal, Sept, 2006 by Teresa M. Heckert, Amanda Latier, Amy Ringwald-Burton, Colleen Drazen

Some professors argue that it is possible to "buy" higher course evaluations by grading more leniently and requiring less work. This study investigated the relations of student effort, appropriateness of class difficulty, and course evaluations with a sample of 463 undergraduates from a mid-sized, Midwestern university. Findings support the treatment of difficulty appropriateness and student effort as distinct concepts. Results refute the grading leniency hypothesis. Consistent with cognitive dissonance theory and the validity hypothesis, student effort was positively related to all dimensions of course evaluation, and these relationships could not be explained by effort's relation to expected grades. Student effort was also significantly related to a number of student, instructor, and course characteristics.

**********

Although many administrators claim that teaching is the most important professor activity, frequently, the only tool used to assess teaching effectiveness is student ratings. Ratings are used to make personnel decisions, including promotion and tenure. Reviewers (c.f., Aleamoni, 1999; Wachtel, 1998) have concluded that ample evidence exists that, in general, supports the validity of these ratings. However, ratings do appear to be susceptible to influence by factors that may be unrelated to teaching effectiveness. One of the most widely studied and contentious of these factors is expected course grade. Some professors argue that it is possible to "buy" higher evaluations by grading more leniently and requiring less work. This grading leniency hypothesis predicts a positive relation between expected grades and evaluations of teaching and a negative relation between student effort and course evaluations.

Many studies have found a positive relation, typically .10 to .30, between expected grades and course evaluations (Feldman, 1997). Aleamoni (1999) reported 24 studies finding no relation and 37 studies finding a positive relation, with a median correlation of .14. Wachtel (1998) concluded, based on his review of the literature, that there was a moderate positive relation between expected grades and evaluations. The inconsistency in the findings of individual studies may stem from using global measures rather than separate dimensions to assess teaching effectiveness (Aleamoni & Thomas, 1980). Course evaluations are multidimensional (Cashin & Downey, 1992; Marsh & Dunkin, 1997), and, as a result, the relation of expected grades to evaluations may vary depending on the dimension (Frey, 1978).

In addition to using global evaluation measures, most studies have tested only half of the grading leniency hypothesis and have not measured student effort. Rather, it is merely assumed that an instructor who grades leniently also expects little of his or her students. For example, Needham (1978) postulated that students give higher evaluations to professors who give the best "grade per unit of student effort" (p. 39); however, he did not empirically test his model. Moritsch and Suter (1988) found a relation between effort and halo error (i.e., the failure to differentiate among rating items) but did not report the relation of effort to the actual evaluation.

Only a few studies could be found in which effort and expected grades were both measured directly. Brodie (1998) found the highest evaluations went to professors with the highest grades for the least amount of student effort. Similarly, Greenwald and Gillmore (1997) found a negative relation between grades and workload, such that higher grades were received in courses with lighter workloads. They also found higher grades resulted in higher student evaluations. Taken together, these findings seem to support the grading leniency hypothesis.

Contrary to the grading leniency hypothesis, cognitive dissonance theory suggests a positive relation between student effort and course evaluations. According to this theory, as effort expended increases, enjoyment of the task increases (Reeve, 2005). Research on effort justification has found support for this premise in attitude change (Linder & Worchel, 1970), completion of boring tasks with an unpleasant and rude experimenter (Rosenfeld, Giacaloni, & Tedeschi, 1984), weight loss (Axsom & Cooper, 1985), group initiation (Aronson & Mills, 1959), and consultation work by managers (Nahavandi & Chesteen, 1989). Applying this theory, professors who require more work, or who at least encourage students to expend more effort, will get higher evaluations as a way to justify this effort.

Consistent with the cognitive dissonance paradigm, Marsh and Roche (2000) re-examined Greenwald and Gillmore's (1997) dataset and found that higher workloads were actually associated with higher student evaluations of teaching. More positive evaluations were received by professors whose classes were perceived as being more difficult, moved at a faster pace, and required a heavier workload. Marsh and Dunkin (1997) provided an alternative context for understanding these findings. According to the validity hypothesis, better grades reflect greater student learning. Additionally, students are likely to evaluate courses more positively if they feel the course was worthwhile, that is, they have learned from the course. Furthermore, one would expect greater learning to occur in those classes that require more work by the students and are seen as being more challenging, provided that the challenge does not outweigh the students' skill level. In these courses, students are likely to be engaged more actively in the class and with the material, thus enhancing their intrinsic motivation to learn. As a result, course evaluations are likely to be higher for courses in which students perceive the difficulty level to be appropriate.