Doing High-Stakes Assessment Right - education

School Administrator, Dec, 2000 by Andrew Porter

Perfect solutions don't exist so let's proceed with student accountability that's aligned and fair

Student achievement scores are said to be improving; Texas, Kentucky and North Carolina are often cited as examples. Leaders in those states say improvement is due in part to the use of high-stakes assessments.

National surveys by Public Agenda repeatedly find that huge percentages of Americans favor raising academic standards and holding students and educators accountable for achievement produced. But all is not well in standards-based reform for which high-stakes assessment has become the linchpin. Education Week recently explored the breadth and depth of a backlash against high-stakes testing. The same topic was even the focus of a lengthy feature in the August issue of the Atlantic Monthly.

For the past half dozen years or so, I have been working closely with states, such as Missouri and Kentucky, and big-city school districts, such as Philadelphia, advising them on the technical issues involved in their assessment and accountability programs. Each is taking a different approach to high-stakes assessment. Each approach has its strengths and weaknesses. In what follows I draw on these firsthand experiences, plus my technical knowledge as a psychometrician to sketch the do's and don'ts of high-stakes assessments.

Two Goals

At the outset, one must ask two basic questions. First, what is the goal of high-stakes assessment in a standards-based reform program? Second, how is high-stakes assessment supposed to work?

Generally, the goals are twofold: to improve student learning of worthwhile academic content and to narrow the achievement gap between students from more affluent and students from less affluent families. (This often is cast as narrowing the gap between white students and students of color.)

Again, there are two parts to how high-stakes assessment is supposed to work. First, students and educators are expected to work harder toward achieving the above goals. Second, educators are expected to work more efficiently and effectively. The main idea is that high-stakes assessments, aligned to ambitious content standards, will direct educators' efforts toward providing a more ambitious curriculum to all students. The intended result is better student achievement on more challenging content.

Many issues are involved in designing and implementing a useful high-stakes assessment program. In its complexity, it is easy to lose sight of the forest for the trees. The complexity can be reduced to three essential criteria for a high-quality high-stakes assessment program.

First, the program must set a good target and the assessment must demand an ambitious, worthwhile curriculum. Second, the program should be symmetric. Schooling at its best involves students and educators working together to produce high levels of student achievement. A high-stakes assessment program should not hold students accountable without holding educators accountable for achievement as well, and the reverse is equally true.

Third, the high-stakes assessment program must be fair. Here is where the concepts of validity and reliability come into play, especially providing students an adequate opportunity to learn what is assessed.

Let's be clear: High-stakes decisions are made whether or not tests are involved. Teachers make decisions as to whether or not students will be retained in grade or get credit for a course. Schools make decisions as to whether or not a student will graduate. States make decisions as to whether or not a school is accredited. Each type of decision is made whether or not information is available from a test administered by the district or state. Robert Hauser, chair of the National Academy of Sciences' Committee on Appropriate Test Use, found that the end of social promotion in American schools began long before recent high-profile calls for using high-stakes assessment to end social promotion.

Setting Good Targets

Standards-based reform, in theory, sets the course for instruction through content standards, but content standards in practice are rarely powerful guides for instruction. They are not sufficiently clear about what is to be taught, to whom or when. Even if they were clear, content standards still might sit on the same shelves, gathering dust. What gives content standards bite are aligned assessments.

But all too often tests are not well aligned to content standards. Even when they are aligned, high-stakes assessment can make the enacted curriculum (what is actually taught) lumpy. Lumpiness in the enacted curriculum happens when parts of the desired curriculum receive excess focus while other equally important content is underemphasized. There are several ways uneven focus can occur. A school district can test only in the subjects of math and English language arts and only at one grade level for elementary school, one grade level for middle school and one grade level for high school. What one does not want is for high-stakes assessment to cause focus on only a few grade levels and only one or two subjects.