Putting Instruction on the Line - education

School Administrator, Dec, 2000 by W. James Popham

In a districtwide experiment, teachers documented gains in their students high-level skills in science and writing

Each spring, in community after community, newspapers glee fully castigate local schools whose students performed poorly on nationally standardized achievement tests. The premise of those newspaper editorials is clear: Poor test scores stem from poor teaching.

Unfortunately, the newspaper pundits who issue these ritual spring assaults don't understand that standardized test scores are not the proper tool for evaluating a school staffs effectiveness.

Standardized tests do provide teachers and parents with a reasonably good estimate of how a particular child's knowledge and skills compare with those of students in the rest of the nation. That information is useful to have. However, standardized tests do not indicate how well teachers teach.

Review the questions on any standardized achievement test and you will realize that students' success on many items depends directly on their out-of-school experiences. This explains why a strong relationship exists between students' socioeconomic status and their scores on standardized achievement tests. Children who come from a stimulus-rich environment tend to do well on these tests; children who come from a stimulus-poor environment tend to do poorly. Trying to determine a school's instructional effectiveness by using a standardized achievement test is like trying to measure temperature with a tablespoon.

Making a Statement

Teachers and administrators in the Department of Defense Dependents Schools in Wuerzburg, Germany, concluded that students' and parents' perceptions of their instructional effectiveness were based almost exclusively on the students' scores on the nationally standardized tests administered each spring to all DoDDS students.

Those scores typically landed in the middle of the national norm tables--in other words, about average. But the educators believed that, in terms of instructional skills, the district's teachers and administrators were actually well above-average. They needed evidence to support that contention.

District educators wanted to determine whether a group of local teachers could promote students' mastery of high-level cognitive skills. The educators decided to select two high-level cognitive skills, then collect pretest and post-test evidence during the first semester of the 1997-98 school year to determine whether Wuerzburg teachers could effectively promote those skills. They chose to promote persuasive writing and interpreting scientific data displays (making sense of scientific figures, tables and graphs).

Performance Tests

District educators developed performance tests for each skill that clarified for teachers how the two skills would be assessed. Both performance tests were designed with instructional decision-making in mind; that is, the rubrics that would be used in scoring students' responses to the tests were fashioned so all of the rubrics' evaluative criteria could be addressed instructionally. In essence, both performance tests were intended to measure students' skill mastery in a way that illuminated instruction.

Four tasks were developed for the performance test assessing students' ability to write persuasively. The performance test for persuasive writing called on students to compose an original piece of persuasive writing in response to a specific task. A rubric, which could be used either analytically or holistically, was developed to score students' compositions. Students had three class periods to respond to a task, thus allowing time for prewriting activities, drafting and revising.

The science performance test consisted of a three-part task in which students were presented with one or more scientific data displays. Students then were asked to respond to three different subtasks: (1) describe in writing the purpose of a designated data display; (2) write three accurate but different statements of a specified nature based on one or more of the data displays; and (3) write a brief interpretive analysis in response to a request calling for a data-based inference. A rubric was provided for scoring each response.

The Wuerzburg science educators who created this performance test assigned different weights to students' responses to the three subtasks so students received more credit for their ability to make an inference-based analysis than to perform the other two subtasks. Four tasks were developed for this performance test.

Instructional Suggestions

The educators prepared general instructional suggestions and specific instructional suggestions for each skill.

The general instructional suggestions, identical for both skills, offered teachers five practical suggestions for promoting students' mastery of the skills measured by all such performance tests. For example, one suggestion was that students obtain a copy of the rubric (perhaps simplified) that was going to be used to determine their mastery of the skill measured by the performance test.