Book review: The tyranny of educational testing and the ethics of responsibility

Journal of Literacy Research, Sep 1999 by Paris, Scott G, Paris, Alison H, Carpenter, Robert D

With this foreground, in chapter three, Murphy reviews multiple-choice, standardized reading tests in the 1990s. She analyzes both external and internal validity arguments for popular reading tests in a manner that is succinct and scholarly. The review points out the numerous decisions made by test publishers in the design and development of test items, scales, and content and, most importantly, the liabilities inherent in these decisions because they are not politically, conceptually, or pragmatically neutral. Current tests, Murphy asserts, often do not meet acceptable standards within the psychometric tradition, much less reflect sensitivity to broader notions of assessment validity. They conclude that the negative aspects of current practices in test design outweigh the benefits.

As they stand, the arguments made by test developers for the tests reviewed do not inspire a great deal of confidence. The choices facing the test makers are obvious. They must significantly modify test development practices so that they are at least in keeping with good psychometric practice (regardless of what we may think of that practice) and ideally should move to keep pace with current validity theory. The preferable path would seem to be to abandon these test development practices in favor of the development of alternatives. Surely we can now do better. (p. 56)

In chapter four, the authors review individual assessments of reading in a fashion that mirrors chapter three. Again, they examine the explicit and implicit arguments for validation of the tests, and again they find the deficiencies that they were expecting to find. For example, they say,"The competing pulls of marketing pressures and psychometric principles create a contradictory set of purposes for individual assessment measures" (p. 62). The authors conclude that popular individual assessments of reading, although they do not use multiple-choice formats, suffer the same uneven and inadequate validation information as group tests. Furthermore, the individual assessments are often used to diagnose special education students or children with reading difficulties. The ironic tragedy is that children with the greatest needs for comprehensive literacy assessment and instruction are tested repeatedly with narrow-minded tests that highlight their low achievement. In turn, these tests can potentially narrow the focus of remediation by providing overwhelming evidence about what children cannot do well, but precious little evidence about how they read or what they read. Moreover, teachers feel pressure to teach to the test, so they spend more time in test preparation activities and less time with the regular reading curriculum or needs of the students (Smith, 1991; Urdan & Paris,1994). Once again, we see how tests control curriculum and instruction, reinforce the status quo in classrooms, and promulgate inequity.

The first half of the book examines existing reading tests and their development and criticizes them for inadequate validation, both psychometric and hermeneutic. With the widespread use of reading tests as the premise, the authors examine the implications of testing practices under the guise of consequential validity, a new form of validity introduced and popularized in the past 10 years, because it takes into account the way people use tests and test data (Linn, Baker, & Dunbar, 1991; Messick,1988). Part III includes two chapters: a selective historical review of the uses of reading tests by Patrick Shannon and an analysis of the effects on stakeholders of standardized tests written by Peter Johnston. Shannon uses illustrative quotations to show how reading and reading tests were co-defined and then co-opted by various psychologists and educators throughout the past century. His selective review reveals how reading became situated in theoretical, cultural, and political values, and his strident position is clear: