Using data to inform instruction: while test scores are accepted as a measure of school effectiveness, the data should also be used in a more student-centric way to inform targeted instruction

Leadership, Nov-Dec, 2004 by Dennis A. Jonston, John T. Lawrence

Welcome to California's Accountability Cafe! Allow me to recommend our soup du jour, Acronym Bisque. The base is complex--tangy with somewhat of a bite, consisting of a blend of AYP, AMOs, API, CAT/6, CSTs, CAHSEE, CAPA and a pinch of CELDT.

With the implementation of the Federal No Child Left Behind legislation, we were all introduced to a variety of acronyms designed to simplify the discourse surrounding the mandate. Similarly, as we continue to use the Academic Performance Index as a state measure of accountability, the opportunity to use shortspeak (as opposed to shorthand) has been afforded us through the coining of terms such as CAHSEE, CAPA and CELDT, to name a few.

These acronyms, like the legislation they represent, are presumably intended to simplify and make more "user-friendly" the complex task of measuring student learning and identifying schools and districts demonstrating annual growth, or what is more commonly referred to as "adequate yearly progress."

Or not.

It should come as no surprise that the ingredients of California's Acronym Bisque, by and large, represent the tests used to evaluate schools and districts. With very few exceptions, the indicators used to identify underperforming schools take the form of multiple-choice test results and student writing samples.

While student test scores are generally accepted as an appropriate proxy for measuring the effectiveness of schools and districts, it has been argued by many that these data should be used in a more student-centric fashion to inform and promote targeted instruction sensitive to the needs of children.

It is this perspective that challenges my colleagues and me to go beyond the carefully crafted acronyms, the mellifluous discourse and the intellectual circumlocution often associated with sweeping legislation and focus on that which, ironically, appears to have received about half the attention afforded the legislation itself--our students.

Having embraced this challenge, the San Diego County Office of Education's Assessment, Accountability and Research Unit provides professional development opportunities that are student-centric with respect to classroom assessment practices, data analysis strategies and the use of test scores to inform instruction targeting individual student needs.

Teacher preparation

Maintaining our focus at the student level, our initial efforts included a workshop designed to provide teachers and administrators with the necessary tools for identifying precisely what their students were learning and what gaps existed.

After working with teachers at all grade levels and content areas, we've come to understand that the majority of teachers have had little or no formal training in the area of classroom assessment and evaluation. In fact, after more than 50 workshops representing more than 2,500 participants, we have yet to have more than 10 percent of an audience indicate they had received a semester-long course on classroom assessment and/or measurement.

Given the high-stakes nature of assessment and accountability in public education today, this critical need in teacher preparation has helped to define our work for the foreseeable future. Our workshop, "Developing Standards-Based Assessments for the Classroom," addresses the various types of tests commonly used in classrooms as well as their respective components.

Participants are introduced to assessment concepts and vocabulary such as content validity, reliability, test-item development, item selection and the use of test results to inform instruction. In a nutshell, this workshop is designed to touch on the major aspects of developing and using standards-based classroom assessments to inform instructional effectiveness, student mastery and remediation efforts.

The framework underlying our work in this area is best reflected in the relationship among curriculum, instruction and assessment. It is the relationship among these three domains--each reflecting the state standards--that provides the foundation for developing targeted classroom tests.

It is our belief that aligning curriculum, instructional practice and assessment strategies is a critical step toward accurate identification of learning and instructional deficits, identifying promising practices and promoting student learning.

I've often stated that teachers should never find themselves in a situation where they're teaching to a test. However, all teachers, in order to evaluate their instructional effectiveness and students' learning, should "test to the teach."

Testing to the teach

This fragment of thought helps characterize the alignment of assessment, curriculum and instruction. Embedded also are a number of important considerations that must be entertained when developing sound, standards-based tests.

First, a clear understanding of the curriculum and instructional resources used to exercise the standards defining the lesson must be demonstrated.

Second, recognition of instructional intensity, or time spent teaching each standard or set of standards, is essential for establishing content validity and appropriate weighting of test items.