Data in Your Hands - collecting and using data for school mangement information systems in New Hampshire

School Administrator, April, 2001 by Raymond Yeagley

Several sessions on decision support systems and how they benefit school districts captured my attention at last year's management information systems conference, sponsored by the National Center for Education Statistics and the Arizona Department of Education. The highlight was a demonstration of a locally developed data warehouse used by the 60,000-student Tucson, Ariz., Unified Schools.

I was impressed with the work this district had done, but it seemed to me that a decision support system would be impractical for my much smaller system in Rochester, N.H., even with the high-end technology available in our schools. Implementation would require additional technical staff beyond the financial capacity of our 4,500-student, low-wealth district.

I was wrong. Within a few weeks, our district was invited to participate in an AASA pilot project using the Quality School Portfolio, developed by the Center for Research in Evaluation, Standards and Student Testing at UCLA under a U.S. Department of Education grant.

QSP is one of several tools available for integrating and analyzing school data (see additional resources, page 8). It is an easy-to-use database that will load records from any source that exports to a text file. Analysis tools in QSP permit multiple levels of disaggregation, provide automatic tracking of goal progress based on the indicators matched to those goals, support cross-sectional and longitudinal analysis and include reporting functions with graphs, tables and "dashboard" displays that make the information clear and easy to understand.

Getting Started

Collecting and organizing data are neither the most important aspects of data use nor the most difficult tasks. Much of the information needed by schools already is available in electronic format.

Administrative software packages used by most schools store student profiles, grades, attendance and discipline records. Testing companies can provide electronic versions of their scoring reports that include student-specific information. Additionally, scanners permit districts to automate tabulation of surveys and other local data collections. Availability and compatibility of data are no longer a barrier.

A greater challenge than collecting data is creating a process to transform the data into easily accessible, useful information that staff members will employ for school improvement. Building on a goal-setting and accountability process already in place in our district, we identified four principles to guide our efforts in Rochester:

* Instructional change is the first priority. Data will be used to identify district, school and classroom strengths and weaknesses, then find ways to reinforce the strengths and address the weaknesses to improve student learning.

* Staff training is essential for effective data use. Staff members must understand not only how to interpret the information accurately, but also how to identify, adapt and apply more effective instructional strategies based on their analysis.

* Communicating results is a vital component. Communicating a complete, accurate and understandable picture of our district's performance to all of our constituents will improve community support and encourage school effectiveness.

* Inviting feedback closes the loop. Obtaining feedback from constituents is as important in assessing district and school progress as measuring student achievement. In addition to the traditional performance indicators, the district can benefit from obtaining and analyzing data on community satisfaction and all other aspects of operation.

Asking Good Questions

One of the most difficult challenges for a district is to conduct a critical self-assessment. Critics and supporters alike often find it easier to start with their conclusions, usually characterized as "logical" or "common sense," then search for evidence to support them. Likewise, observers frequently limit their examination to data that already are available.

Both of these approaches reduce the value and utility of the inquiry. A more productive approach is to start with probing questions that will get at the heart of the district's performance, then find the data and new ways to assess the data that will answer those questions.

Our school district has suffered from these limitations for years. We have looked at item analyses on the state and national assessments to see what questions were missed most frequently by our students. We have studied individual student results to identify missing skills and have provided extra help in those areas, as time and resources permitted. We also have looked at the district-wide averages to see whether the trends were headed in the right direction. While this approach has helped some students, it has been of little value for systemic instructional improvement.

A database like the Quality School Portfolio encourages the broader inquiry by integrating information from a variety of sources to identify previously hidden patterns in student learning and other aspects of school operations. This allows educators to address needs systematically, instead of on a student-by-student basis.