Improving schooling for language-minority children: A research agenda

Bilingual Research Journal, Spring 1997 by Julie Maxwell-Jolly, Patricia Gandara

It is the view of the Report's authors that studies of effective and nominated schools and classroom designs have added to the literature by providing "rich and highly detailed descriptions, some quantitative and some qualitative, of schools and classrooms" and "researchers have become increasingly sophisticated in the range and depth of data they collect. The resulting rich portraits of supposedly effective programs and practices are especially welcome given the growing diversity of the U.S. school population" (pp. 187-188). Notwithstanding, the authors of the Report observe that there are limitations to what these studies can tell us. First, the studies do not adequately address the question of how a school or classroom became effective, Second, it is difficult separating cause from effect in effective and nominated schools studies which generally make strong claims of causal relationships: "do the characteristics of schools cause them to be effective, or does effectiveness lead to these characteristics?" (p. 188). Third, the nominated schools design in current favor includes no data on student outcomes, even when such data is used in the selection process. The Report suggests that these studies "by definition" are inconclusive regarding effectiveness since schools are often chosen whether or not researchers can link school characteristics to positive outcomes. For example, in the exemplary programs research, schools are often nominated as "exemplary" because they fit researchers' and practitioners' beliefs and philosophies, rather than because they achieve desirable student outcomes. Some of these studies also exhibit methodological problems: some studies describe only a single school, making it difficult to draw any conclusions about the relative importance of given features and others fail to adequately describe data collection methods, have limited depth and duration of observations, or lack systematic methods for observing classroom events.

The Report notes that the same questions of validity raised in regard to the effective and nominated schools research, i.e., "How do we separate cause from effect?" apply to case study research, which focuses exclusively on one intervention site. "How do we know that shifts in achievement patterns following implementation of the changes are due to those changes? What about extraneous factors, general upward drift in scores, or regression effects?" (p.190).

Finally, the Report touches on methodological concerns with some quasiexperimental studies. A feature which gives strength to this design is the comparison group of students who are matched on a number of characteristics, thus allowing researchers to make causal inferences regarding the relationship between school features and student outcomes. However, the authors remind us that, "School changes are so complex and involve so many dimensions that it is usually very difficult to draw tight linkages between specific processes or program components and student outcomes" (p. 190). In our opinion, this may be the greatest understatement of the volume. The Report also notes, "Many of the quasi-experimental studies also have the limitation of providing very little information about the intervention received by the control group, thus making it difficult to gauge the actual merits of the intervention" (p. 190). However, while the authors note that smaller scale studies are often limited by these methodological problems, they nonetheless have contributed significantly to the understanding of process and hold important potential for the development of a strong research base in this field.