Multiple measures: Beginning with ends: One illinois district uses more than scores to assess its students and schools - education

School Administrator, Dec, 2001 by Fred C. Schroeder, Sally Pryor

It is a miracle of the Internet--a miracle born of one school district's refusal to accept simple solutions to complex problems.

Our problem was one many school districts face: How do we define what we want our students to know and be able to do at the end of their journey with us? How do we determine what kinds of adults we hope they will be? What value will we add to their learning and their lives? How do we establish targets for quality improvement? Are there deeper questions that need to be considered as we serve the needs of the 4,200 students in K-8 in our suburban Chicago school district?

These were the questions that our board of education struggled with as it began to ask what lay beyond test scores.

A Composite Index

A major step toward a resolution of the dilemma came when an Internet search turned up an article in the May 2000 issue of The Elementary School Journal titled "Toward a Composite Index of School Performance," which addressed the question of what students should know and be able to do. The author, Richard Rothstein, argued that although "dialogue about schools focuses mostly on student test scores in reading and math, and to a lesser degree, science ... American leaders, citizens and parents have repeatedly emphasized that a broader array of outcomes and processes comprise schools' purposes."

Rothstein introduced a concept he called a "composite index of school performance" to address this issue. He added: "School controversies focus mostly on academic test scores. But Americans insist that schools have broader purposes, including also equity of outcome, citizenship, social ethics and health, and Americans want children to be happy in schools."

A nice statement, yet Rothstein moved beyond the rhetoric to propose a model that defines dozens of possible components to the index, grouped in four areas: academic outcomes, non-academic outcomes, processes and consumption measures. His index included the predictable competencies in the core academic areas, but also included ideas such as wellness and social ethics. He suggested looking at teacher quality and parental involvement as well as school facilities, class size and freedom from violence. He further suggested that perhaps it was important for weights to be assigned to these factors.

Rothstein's premise was intriguing. However, he cautioned that many of the factors, perhaps even most of the factors, were beyond the ability of individual schools, school districts or state agencies to control. Nevertheless, our interest was piqued. Could the concept be applied to a single school district? We were not sure, but we were ready to begin the journey. Our school district theme in 1999 had been "If Not Us, Who?" A year later, it became "If Not Now, When?" and in 2001, "Serving the Needs of the Whole Child--Looking Beyond ... ." Easy words to say but tough concepts to make real.

If finding Rothstein's article was a miracle of the Internet, it was just the beginning. We were prepared to let the Internet also help us find Rothstein. Our search engine quickly located him. We found Rothstein was a research associate of the Economic Policy Institute in Washington, D.C., and an adjunct professor of public policy at Occidental College in Los Angeles. It did not take long to locate his e-mail address and fire off a message. To our great surprise, he responded within hours and began a dialogue on our mutual interests.

Rothstein appeared interested in what our district was doing. He came to realize we were genuinely interested in looking beyond standardized test scores, and we realized his composite index could form a template on which to build a series of measures that could be shaped to our district's priorities.

After several e-mails and conference calls, Rothstein visited our district. He met with board members at an early morning breakfast meeting and joined us for another meeting that night. His comments and observations proved both encouraging and disappointing. Rothstein told our board members that what they were attempting to do was difficult. And although many school districts grapple with these issues, our board's commitment to go beyond test scores alone as a measure of success was remarkable and to his knowledge unique. He cautioned, however, that this endeavor was fraught with obstacles. Board members would have to use care that the goals they adopted truly reflected community values and that the community would accept this approach.

Rothstein visited our middle schools, talked with teachers and asked many questions. Out of these visits he wrote one of his weekly Lessons columns in The New York Times. Titled "A School District Refuses to Worship Scores Alone," it described our interest, spotlighted our attempts and encouraged us further.

A Broader Portrait

Looking beyond performance on standardized testing had been discussed in our district since at least 1997. At that time, four school board members managed to be re-elected in spite of a challenge that our test scores were not high enough. The successful candidates acknowledged that test scores did not tell the whole story. As one board member asked, "What is the rest of the story?" Another wondered if we should spend more time addressing bullying at our middle schools than trying to push up our test scores? The concept of "educational ends" was born.