Data-driven districts: four districts that take different tacks using data to inform key decisions

School Administrator, Dec, 2002 by Scott Lafee

In business and industry, data are king. Information about customers, inventory, sales, rates of return and employee turnover is crucial. Good data help determine success or failure.

Education, of course, is a different animal. Its essential product is children, who aren't at all like mass-produced widgets or numbers on a spreadsheet. Kids are real, data are abstract--and for some educators that makes it a four-letter word.

"Not many educators buy into or pay much attention to numbers in graduate school," says Philip A. Streifer, a former superintendent and now an associate professor of educational leadership at the University of Connecticut. "Dealing with data is a subject that makes people uncomfortable. It causes a good deal of anxiety."

Yet the school reform movement increasingly demands just that. The call for greater accountability means administrators and teachers must show proof--tangible, statistically valid evidence--that what they are doing is working, that students are learning faster and better.

Rapid Spread

The old tools of education--intuition, teaching philosophy, personal experience--do not seem to be enough anymore. Virtually every state has put into place an assessment system intended to measure and validate student achievement and school performance.

The passage of the No Child Left Behind Act only raises the stakes further, adding new responsibilities for states, districts and schools to accurately and promptly collect, analyze and report data on student demographics and achievement. Federal law now requires annual state reports on student performance by gender, race, disability, income, migrant status and English fluency. States also must report how many students were not tested, in the same categories, and provide detailed information about the professional qualifications of those teaching the students, such as the percentage of classes taught by teachers lacking certification in their subjects.

"Right or wrong, external accountability is coming to everyone," says Katherine Gemberling, an educational consultant and former deputy superintendent in Montgomery County, Md. "You can't simply mandate educational quality and order up tests to make sure it happens. But the fact is, external accountability models exist because educators did not step up themselves and establish definable measurements of quality. ... Educators feel compelled--they are compelled--to look at anything that will help them show they're getting good results."

In other words, like good businesspeople and well-run companies, educators and school districts are being asked, if not expected, to prove their bottom line with hard, solid data. With passage of No Child Left Behind, school districts large and small have taken up the banner of data-driven decision making.

"We have seen an increased interest by districts in DDDM," says Geannie Wells, former director of AASA's Center for Accountability Solutions. "We saw the momentum building before No Child Left Behind and then definitely a heightened interest after."

But no revolution runs smoothly or easily. There are always missteps. Though districts and states generate huge amounts of data, many remain ill equipped to make the most of it, to effectively employ that information in policymaking or in reporting to federal education officials.

"Ownership of data has been a problem," Wells says. "Some data reside at the state level, the district level, the school level or even with vendors in proprietary formulas. Incompatibility of various data systems has been a problem. Often enrollment data can't be tracked with student achievement data or program participation data, etc. Inadequate funding and technical support for data collection have been challenging. And data from testing companies and states are often not disaggregated to such a level that they can be useful and are not available in a timely fashion."

Only 16 states, for example, have systems that give each student a unique identification number that allows them to be tracked and assessed over their entire school career. Only 17 states provide demographic breakdowns of student achievement by race or family income on school report cards.

Slow Progress

"Everybody isn't starting at the same place," Gemberling says. "Everybody doesn't have the same tools, whether that's the technological infrastructure or the people trained to handle it. ... You can't just crunch some numbers and expect that this will lead to effective, real-world decisions. How, for example, do you measure growth, teacher effect, student ability? What is evidence of quality? Good data about these things require more than just standardized test results."

Streifer knows those difficulties well. As superintendent of the Avon Public Schools in Connecticut in the 1990s, he came face-to-face with the hard realities of data--or lack thereof.

Like most school districts, Avon collected lots of information. But much of it was organized or reported in ways that rendered it hard to use or worse. "Data collection is a messy, messy business," Streifer says. "It's done in different formats, sometimes electronically, sometimes on cards or paper. Often it's incomplete. Teachers collect it differently, and not everybody has the same access to it, which means not everybody is going to be on the same page."