When raising isn't rising: the failure of accountability systems to measure student growth over time

School Administrator, Dec, 2002 by Dennie Palmer Wolf

Ask any educator in the hallway outside your door: "So what is standards-based reform all about?" In all likelihood, the answer will be something to do with "raising expectations" or "all children can learn to high levels."

In one sense your informants are correct: American educators and policymakers have hashed out the ongoing importance of proficient calculation, defined the role of phonics in balanced literacy and declared an uneasy truce about whose history must be taught. Most also agree that two years of business math is not enough for a graduating high school senior. An 8th grader should be able to write a coherent and persuasive essay. And 3rd graders should read fluently and critically.

Having raised the standards, educational decision makers have moved on to raising the stakes for failing to meet these higher expectations. For students, the consequences run from mandatory summer school to no high school diploma without passing the state test. Likewise, increasing numbers of teachers work in systems where either the school or the individual teacher may receive cash awards based on student performance. At the school level, a staff faces clear benchmarks to hit as well as costs for failing that range from technical assistance to reconstitution or closing.

A harsh or even just a candid critic of the standards-based school reform could say, "Right, OK, so the adults can sleep at night. They 'duked it out' and got the standards and consequences down on paper. Now about the children ...?" This is more than casual irony. Behind the remark lies the most fundamental question in school improvement: Having invested heavily in 'raising' both the standards and the stakes, what investment are we willing to make to support students in 'rising' to meet those standards?

The Dominant Model

The newly enacted Elementary and Secondary Education Act of 2002 (No Child Left Behind or NCLB) puts this issue of rising to meet the standards at the forefront of its framework. The law explicitly requires using annual individual testing of children in grades 3-8 in mathematics and literacy to drive accountability for changing the level of student performance.

Specifically, the NCLB legislation requires schools to establish a base-line performance and then to show "adequate yearly progress" for each of 10 successive years, with the goal of 100 percent of children performing at the proficient level in mathematics and reading. As with any vision, the devil is in the details. The details of NCLB all reflect a specific model of accountability, a model that reigns in districts and states as well.

To illustrate this point:

* The legislation is entirely based on the use of data from standardized tests. While such tests provide an efficient and reliable measure of some aspects of student achievement, as currently formulated, few of the widely used tests probe students' mastery of complex or high-end skills: developing a finished, as opposed to a first-draft essay; interpreting data from an experiment or translating a conversation into another language.

* Testing is designed to look at student achievement at a particular point in time. Even though NCLB testing will soon be yearly, the tests and analyses examine the variance within student performance at a specific grade level. The fundamental questions are: What percentage of 4th graders score at basic, proficient and advanced levels? And how is this pattern of scores different from those of last year's 4th graders? The basic approach is to conduct repeated comparisons of specific years of performance--not successive years of individual or groups of children's achievement.

* Just how test score gains defined in this way fit into mastering the standards is unclear. For instance, it is an open question as to how many years of gain can be covered simply by helping children to answer correctly increasing numbers of relatively low-level items without substantially changing children's command of fundamental concepts or important strategies.

* The NCLB legislation addresses the problem of improving student achievement and closing the gap as if every interval of change were the same. Yet early gains likely can be achieved through teaching the format of the test, while later gains can be achieved by responsibly teaching basic skills. By contrast, we have little knowledge about teaching the thinking skills or cultural capital that will be the substance of later gains.

Competing Models

These are substantial issues, in and of themselves. But a still more fundamental problem exists. As suggested above, the accountability mechanisms proposed for NCLB use an attainment, as compared to a development, model of accountability

In attainment models, no one asks about the longitudinal history of current scores. The danger of this is readily apparent. This year's 8th graders may outstrip last year's 8th graders, but they may be substantially underperforming relative to what they accomplished in 7th grade. Potentially, the school or the district has a bold and broadly successful middle school program for 6th and 7th graders that dead-ends into a test-driven final year that endangers students' transition to high school.