Fixing federal research: education demands a first-rate R&D shop. The Department of Education isn't it-yet - Feature - research and development

Education Next, Winter, 2001 by Maris A. Vinovskis

Federal funding and direction routinely lead to research and development breakthroughs in science, medicine, and defense, among other fields. Witness the recent debate over federal funding of stem-cell research, during which supporters of stem-cell research emphasized that federal funding tends to draw the best scholars to a field. So why do research programs funded by the U.S. Department of Education continue to disseminate research that is barely respected by serious scholars? Many academics and policy-makers consider the education research sponsored by the National Science Foundation or the National Institutes of Health to be more rigorous and scientifically sound than the research emanating from the Department of Education's Office of Educational Research and Improvement (OERI). Likewise, the department's Planning and Evaluation Service (PES), tasked with evaluating the performance of federal education programs like Title I, has a mixed record of developing sound and timely information on program quality.

As policymakers look to redesign or replace many of the basic federal education programs established during the 1960s, they face mounds of Department-produced research to guide them, but much of it is of limited value. This is a problem that has cut across the decades, during both Republican and Democratic administrations. With OERI up for reauthorization by Congress, this seems like the perfect time to revisit the federal role in research and development.

The Centers and Labs

Federal education research and development has been fragmented and disorganized for most of its history. In an effort to address these problems, in the mid-1960s the government created two complementary sets of institutions intended to promote more coordinated, long-term, large-scale education research and development: the Research and Development Centers and the Regional Educational Laboratories. However, most of their work turned out to be (and continues to be) short term and small scale. They tend to focus on small, individual research projects rather than supporting large-scale development and evaluations of alternative techniques and programs for helping all children to succeed in school.

The value of their research and development efforts has been routinely criticized, yet the share of research funding devoted to the centers and laboratories rose under both the National Institute of Education and its successor, OERI. The labs used their lobbying arm, the Council for Educational Development and Research, to protect their budgets even as funding for research proposed by individual scholars and other initiatives was disappearing or was dramatically curtailed during the Reagan years. As funding for OERI grew through the 1990s, a large share continued to go to the centers and labs or to activities only loosely related to research or development, such as distributing funds for introducing more technology in the classrooms (see Figure 1).

Neither the recent OERI third-year assessment of the centers nor the PES third-year review of the labs was a thorough or totally objective evaluation of the R&D contributions of these institutions. The results were also not made available to the public or to policymakers in a timely or convenient manner. Yet in the closing months of the Clinton administration, OERI made a last-minute decision to fund some of the existing centers for another five years and offered new five-year contracts to the labs--when the agency hasn't even been reauthorized by Congress yet. It might make some sense to extend existing center or lab grants for another year while awaiting OERI reauthorization, but the decision to make long-term commitments to these institutions, based on inadequate evaluations and before Congress has even signaled its willingness to continue to fund them, seems quite misguided.

To be sure, the sheer paucity of funding places a serious limit on the quality of federally funded education research and evaluation, yet this hardly explains all the problems of education research and evaluation today. OERI has frequently suffered from unstable and intellectually weak leadership and the absence of a distinguished research staff. After Sharon Robinson completed three-and-a-half years as head of OERI, the agency went through four different heads in less than a year. During the Clinton administration, OERI personnel were cut by 25 percent, worsening the agency's shortage of capable, innovative researchers. Furthermore, the agency has not taken advantage of its opportunities to hire qualified researchers. In the waning days of the Clinton administration, OERI appointed full-time directors for its five research institutes, but only one of these individuals has extensive research training and scholarly experience.

As a result, neither OERI nor Planning and Evaluation Service is providing a significant number of scientifically sound and educationally relevant program evaluations, The education marketplace is crowded with competing "whole school reform" models, such as Success for All, Accelerated Schools, and Core Knowledge, all promising remarkable gains in student achievement, especially among disadvantaged students. If one were to define the federal government's role in education, wouldn't one of its core functions be studying whether any of these programs actually live up to their claims? OERI has yet to systematically examine which models of education interventions are especially effective at improving student achievement in different settings. Nor has PES, which has the primary responsibility for conducting program evaluations, been able to accomplish this important task. Consequently, school districts lack the kind of research-based evidence necessary to choose among such programs; often the only information they have comes from the program developers themselves. Even worse, the federal government, through the Obey-Porter legislation, is giving aid to districts to implement many of these programs even though few of them are backed by any solid research.