Assessing quality in higher education

Liberal Education, Spring, 2001 by Douglas C. Bennett

How CAN WE ASSESS the quality of education offered by a college or university? How can we know reliably whether or when learning is taking place?

How can a prospective student evaluate whether she will get a good education at an institution where she is considering enrolling? How can a parent have confidence that his son or daughter is learning at the college to which he writes tuition checks? How can a governor or legislator come to terms with the effectiveness of the education offered within a state? How can a faculty assess the strengths and weaknesses of the educational program it offers?

No questions could be more important. And yet we ignore them or (just as bad) accept shallow or misleading answers. Thus, these questions are embarrassing ones. The National Center for Public Policy and Higher Education recently released Measuring Up 2000: The State-By-State Report Card for Higher Education. The study uses systematic data to prepare a report card for each state on its higher education system in terms of six categories: preparation, participation, affordability, completion, benefits, and learning. Every state received a grade in each category. But every state received an Incomplete for learning. No state has yet developed an adequate approach to assessing student learning. And yet surely this is the most important category of all. This national report card tells us a great deal about higher education in the states, except whether their institutions are fulfilling their undergraduate mission.

Value added: The only valid measure

Virtually everyone who has thought carefully about the question of assessing quality in higher education agrees that "value added" is the only valid approach. By value added we mean what is improved about students' capabilities or knowledge as a consequence of their education at a particular college or university. Measuring value requires having assessments of students' development or attainments as they begin college, and assessments of those same students after they have had the full benefit of their education at the college. Value added is the difference between their attainments when they have completed their education and what they had already attained by the time they began. Value added is the difference a college makes in their education.

Easy as it is to state, assessment of value added is difficult to carry through. Let me briefly mention just a few of the more important difficulties.

* Value has many dimensions. No college or university is trying to develop only a single capability in students; all are trying to develop an array of capabilities. Measurements of value added must therefore attend to a number of different dimensions of value. We probably should develop several different measures of value added and invite institutions to select the measures that reflect their intentions.

* Institutions are different. Colleges and universities do not all seek to add the same kind of value to students' development.

Even liberal arts colleges do not all have the same mission. We need to assess value added against a college's chosen aspirations--its mission. Any effort to rank colleges or universities along a single dimension is fundamentally misguided.

* Effects unfold. Some consequences of a college education may take years to express themselves. We may need to assess some aspects of value added with alumni rather than with graduating seniors.

* Complexity and Cost. Measurement of value added is likely to be complex and expensive. Yet it can be more expensive for society to have no serious assessments of whether we are succeeding in having students learn.

A value-added approach is the best way to assess student learning, but higher education has not yet committed itself to developing reliable measures of the most important dimensions of a college education. There are, on the other hand, a few other possible strategies for assessing student learning that are worth considering.

Assessing outcomes: A second-best strategy

A second strategy for assessing quality is simply to measure the outcomes of a college education: evaluate students as they graduate (or shortly after) on the skills and capabilities they have acquired or the recognition they gain in further competition.

It is possible, for example, to look at GRE scores for those students who take GREs, or to measure the percentage of students who go on to further graduate study, or to look at the honors won (Rhodes, Watson, Fulbright) by graduates. At best, these measures evaluate the quality of an institution's best graduates, not the attainments of all its graduates.

The most frequently used outcome indicator at present is the measurement of retention rates. What percentage of those admitted to a particular institution continue in the program or finally earn a degree? Retention rates tell us what percentage of an institution's students were satisfied enough to continue at a college, and what percentage received the benefit of the institution's full program. But they do not tell us anything about what students actually learned or attained on their way to a degree.