Surveying university student standards in economics

Economic Papers (Economic Society of Australia), June, 2005 by Peter Abelson

4 Issues in a Survey of Student Standards

Needless to say, many issues arise in a survey of student standards. Some issues are fundamental. They include such questions as What is quality? What is evidence of quality? What incentives do department heads face when responding to such surveys? How can we tell whether the responses are honest and accurate? Other issues are more pragmatic. They concern the structure, conduct, and analysis of such a survey. In this section, I first discuss some pragmatic issues before turning to more fundamental ones.

4.1 Some Pragmatic Issues

It may be noted that in preparing the responses we counted each response as one, regardless of the size of the institution. This provides a clearer picture than attempting to weight the responses by, say, the number of economic students in an institution. In any case, the exact numbers in each institution are not known. But this process may give too much weight to the views of respondents from institutions with fewer students.

One reason that the numbers are not clear relates to the difficulty of defining economic students or students in economics courses. Universities have different descriptions of such students and we could not offer a single definition of economics students that all respondents would understand unambiguously. Fortunately this did not matter because we were interested only in approximate numbers of students to ensure that the responses could be understood.

A similar definitional problem arises with student/staff ratios. With increasing numbers of part-time students and staff, student/staff ratios are less precise concepts than they used to be and comparisons over time and across institutions have to be made cautiously. However, the reported student/staff ratios were generally so high that the implications for potential under-resourcing were clear, despite some imprecision in the measure. Other measures of resources per student may be desirable, but are even harder to obtain.

Another basic issue is the heterogeneity of the sub-markets within commerce faculties. As noted above, the Society attempted to deal with this by identifying five separate student categories (first- and third-year undergraduates, Honours, coursework Masters and PhD students). This lengthened the questionnaire considerably and three of the eight non-respondents claimed that they did not respond because of the excessive length of the questionnaire. Evidently there is a trade-off between the length and detail of the survey and the response rate.

Inevitably the choice of questions was selective. A reviewer suggested that, in asking about determinants of standards, the survey should have included such factors as teaching standards, pastoral care of students, student achievement motivation, and learning style. Arguably, student achievement motivation was included implicitly in student work-hours. More generally as noted, respondents were given many opportunities to express their views on issues that were omitted in the questionnaire.