Neglected Evidence - educational research

Washington Monthly, Nov, 1998 by Richard Whitmire

Why doesn't anyone pay attention to educational research?

Two years ago five mid-level Kentucky education specialists sat down in their sixth floor Frankfort office and launched a mission that appeared both reasonable and practical: Find an algebra program with real proof that it raises student performance by 7 percent a year over 3 years.

A reasonable mission statement. But they couldn't find a single curricula publisher who had the proof. "Teachers like our curricula," the publishers would answer when asked for proof. "It's teacher friendly." And it wasn't just that the major U.S. textbook publishers couldn't prove their pricey curricula actually worked. They were offended by the question. "The conversations frequently deteriorated," says Kentucky team leader Jo O'Brien. "They would become snippety." If you have a problem, they would tell O'Brien, check with your own bid list people in the State Education Department. They approved us.

Alarmed at what she was hearing, O'Brien checked with the state purchasing officials, those who assemble the approved vendors on the bid list. Well, actually we don't ask for proof these programs work, she was told. We want to know if the textbook glue will hold up for years. We want to know if their table of contents shows all the topics our state regulations insist they show. Finding little success in algebra, the team expanded its investigation and started phoning curricula specialists in other states. Surely, they thought, other states had been down the same path. Nope. What they heard from the curricula specialists were a lot of anecdotes about their favorite programs. "I just love Montessori," a reading specialist would say. But no proof of success was offered.

O'Brien's team finally faced the facts. They had to be pioneers. They had to explore the uncharted universe of finding education programs with proof they work. Keep in mind, this is 15 years after "A Nation At Risk" was published, the expose of America's educational failures that sent this country on a well-deserved education reform binge. After sifting through 500 programs, they came up with a mere 50. Absent from that list are some of the biggest names in textbook publishing, the giants whose textbooks undoubtedly are passed out in your child's classrooms.

Even today, after their "Results-Based Programs Directory" has been published, O'Brien shivers at the thought of a gigantic education industry never asked to prove that its products work. Instead, it caters to the ever-changing whims of teachers and self-styled education gurus. Let's try the "whole language" approach to reading; let's try organizing middle schools into interdisciplinary teams; let's try tossing out teacher lectures; let's try pulling poor kids out of classes for remedial instruction. Long before any of these methods could be tested to see if they helped students learn, they were scrapped in favor of a newer, shinier model. "We are amazingly lax consumers," says O'Brien. "We've told the industry what matters, and for a long time that was teacher comfort. We don't question the product enough."

Try it, You'll Like It

Paul Hill, an education researcher at the University of Washington, sees parallels between education practices and medicine as practiced in the Colonial period. "There was not a lot of science behind medicine then," says Hill. "It was swept by fads." Each patient's suffering was thought of as unique, a special case. There was no broad research that demanded standardized responses to standard ills. Each patient was in need of individualized care. Ever hear one of your child's teachers use that word? We provide individualized instruction. Each class is unique. "Physicians used to say the same thing," says Hill. "There was a natural tendency to think of everything as much too unique to want to generalize your practice."

But while medicine has moved into an era of standardized practice driven by medical research, education remains in the era of leeches and bleeding. Where was the research that demanded teachers drop phonics, teach in "open" classrooms or try "new" math? Where, for example, was the research that led so many elementary school teachers to tinker with "heterogeneous grouping," where children of mixed abilities were put into groups with the hope that the faster learners would tow along the slower learners? Like so many educational fads, it sounded like a good idea, but it rarely worked in crowded classrooms lacking the talented aides who could pull it off.

At Harvard, former teacher Tom Loveless teaches a subject on this very topic: "Controversies in Education Reform." The syllabus, which consists entirely of required readings on school failures, resembles an indictment of a Mafia chief. Loveless has a personal feel for the problem that dates back to his nine years of teaching in Sacramento public schools. "This is an industry with tremendous turnover at school sites," says Loveless. "Half the principals change schools every six or seven years and superintendents even more frequently. I went through three or four principals, all of them saying: "I have some new ideas and we're going to change things." In come the big changes, out goes the principal within a few years, in come more big changes. "Nobody would stay in one place long enough to be responsible for outcome," says Loveless. "By the time everyone figures out what they're doing doesn't work the principal is gone and you're off to a new approach."