An investigation of school desegregation and its effects on black student achievement

Thomas R. Ascik

The National Institute of Education (NIE) has just published a collection of papers that will shed a great deal of light on one of the great social science issues of our times, namely, the issue of the effects of school desegregation on black student achievement. In addition, it may establish new standards for the use of social science research in the arena of public policy.

The importance of the issue to

public policy

Although the political history of the relationship betwen desegregation and black student achievement has yet to be written, there can be no doubt about the importance of this relationship--to the federal courts, to officials at all levels of government, to school officials, to social scientists, and to parents. The public interest in this issue was launched,of course, by the Supreme Court's decision in Brown v. Board of Education (1954); the Social Science Statement submitted to the Court on behalf of the Brown plaintiffs by social psychologists Isidor Chein, Kenneth Clark, and Stuart Cook; and the Supreme Court's own psychological musings in the text of the brown decision along with its judicial notice of the authority of social science research in the now famous Footnote eleven of the decision.

The Supreme Court had to make three key decisions to arrive at its holdings in Brown. The first was its decision not simply to overturn its ruling in Plessy v. Ferguson (1896) that "separate but equal" facilities for the two races were constitutionally permissible. The second was its decision to issue a ruling about racial inequality in education regardless of whether "the physical facilities and other 'tangible' factors" were unequal. The third was its finding that the history and meaning of the Fourteenth Amendment--the amendment upon which the holding would rest--were "inconclusive." With this admission that the reasons for a strictly legal holding in the case were "inconclusive," the Court foreclosed such a holding. And when it said in its most important utterance in the case that "separate educational facilities are inherently unequal," it meant not that they were unequal in the allocation of measurable, "tangible" educational resources, including "buildings, curricula, qualifications and salaries of teachers," but that they were unequal because racial separation itself causes an unequal effect on the self-esteem of children. And the resultant "sense of inferiority" caused by the low esteem of black children affected "the motivation of a child to learn" and retarded "the educational and mental development of Negro children."

The issue was self-esteem in education. A low esteem affected the educational performance of black children. Norman Miller, one of the NIE authors, remarked in his paper that the Social Science Statement made the same connection between self-esteem and academic achievement, for it "clearly emphasized a casual pattern in which personality variables (self-concept and achievement motivation) caused subsequent changes in academic performance." It has been argued by Jack Greenberg, one of the attorneys for the plaintiffs in Brown, that "since everyone looks to these results [i.e., test results], they should not be an impediment" to desegregation.

In federal court decisions since Brown, black academic achievement has taken on a more explicit role. In the late 1960's and early 1970's, the lower federal courts began to regard achievement test scores as evidence of unequal educational opportunity. For several years, the Supreme Court itself did not comment on such evidence, although the "facts" of the cases it was reviewing were sometimes based on test scores. However, inthe 1977 case of Milliken v. Bradley, the Court dramatically expanded the scope of judicial remedies in desegregation cases from mere student assignment and re-assignment to the prescription of remedial educational programs. This case made the academic performance of black students a primary concern in the fashioning of judicial remedies for segregation offenses. Indeed, in the many urban areas where resegregation has occurred, the question of racial balance and busing has become moot, and judges have increasingly turned to educational remedies for segregative acts.

Prior research on school

desegregation and

black student achievement

The first large-scale study of the effects of school segregation (and desegregation) on the academic performance of black students was the landmark Coleman Report (1966), the intended political purpuses of which were to document statistically the view of the U.S. Office of Education that there were huge differences in the quality of white and black schools and in black and white achievement. Six hundred thousand students and sixty thousand teachers took verbal and mathematics tests and answered questionnaires about their family and economic backgrounds. The well-known conclusion of the report, that student achievement was related more to family background than to either race or schooling, contradicted expectations and has had a larger impact on education than any other piece of social science research. One of its more important effects has been its introduction into desegregation litigation.

The Coleman Report was based on simultaneous achievement testing in a large number of schools, and it has continued to be unique, both for its size and methodology, in desegregation research. More frequently, in order to get an overall idea of the effects of desegregation, researchers have conducted systematic reviews of meta-analyses of a large number of studies. The first major effort to use this assessment technique was Meyer Weinberg's review of the entire field of desegregation literature (1970). Weinberg found that desegregation significantly improved black student achievement, but he made little attempt to screen studies for scientific rigor. This liability applied as well to Weinberg's later review (1977) on the same subject. Nancy St. John (1975) undertook a review of forty-three studies of the effects of school desegregation on black student achievement. She grouped studies according to their research design, thus permitting a comparison between the methodology of a study and its conclusion about the size of the effects. She found that the more rigorous studies showed smaller positive effects for black improvement, but that, overall, the evidence was mixed. L.A. Bradley and G.M. Bradley (1977) conducted another all-inclusive analysis of the research literature and concluded that the body of studies as a whole suffered from so many methodological flaws and lapses of rigor that nothing definitive could be said about the effect of desegregation. In 1978, Ronald Krol used the recently-developed techniques of meta-analysis to screen fifty studies for minimal rigor and to make a determination of the effects size. He found that desegregation has a positive effect on black achievement but that effect was not statistically significant. In 1982, Robert Crain and Rita Manhard used meta-analysis to scrutinize ninety-three studies and found an insignificant improvement in achievement overall but a significant improvement when desegregation was carried out on a metropolitan basis before or at the first grade. Paul Wortman (1983) criticized Crain and Mahard's effort for allowing the inclusion of too many studies with serious methodological flaws. Wortman conducted a meta-analysis of thirty-one studies that he thought met minimum methodological standards, and he concluded that desegregation caused significant improvements in black student achievement.

BEsides these reviews, Harold Gerard and norman Miller in 1975 published the findings of their longitudinal study which had involved the largest collection of original data on desegregation since the Coleman Report. They followed the personal histories of 1800 children over a six-year period in a single school district undergoing desegregation, but were unable to conclude that minority achievement rose after desegregation.

The NIE project

Because of the importance of the issue for public policy, the disagreements among recognized experts, and the recurrent question about the quality of research in the field, the National Institute of Education decided to see if there was a way to reconcile the disagreements and clarify what social science research has to say about the relationship between desegregation and black student achievement.

It was decided to convene a panel of seven highly reputable social scientists who had done considerable work in the field and who had strong reputations for methodological rigor. The line-up would roughly comprise two social scientists who had concluded from their studies that desegregation had positive effects on black achievement, two who had concluded that it had negative effects, and two who concluded that the effects were neutral, with the seventh as a kind of methodological policeman and commentator on the other six papers. The possibility of making prior bias less likely by choosing seven social scientists who had never studied desegregation was considered but rejected as probably unworkable because of the huge amount of time it would take for such a panel to acquaint itself with the field. Selected for the panel were Robert Crain and Paul Wortman (positive effects), Norman Miller and David Armor (negative), and Herbert Walberg and Walter Stephen (neutral). Thomas Cook was the seventh member. Characterizing the prior work of these researchers as positive, negative, or neutral was not meat to label them but was only a useful device to insure a diverse and comprehensive panel. This balancing of prior viewpoints was also done to pre-empt charges of ideological or political bias. Two other aspects of the project argued against such labelings: the papers were an opportunity for each to demonstrate--and possibly revise--what his own position really was, and, more important, the emphasis of the work of the panel was to be on scientific objectivity, that is, on what rigorous research requires one to say about the effects of desegregation on black achievement. Because of this, the prior conclusions of the panelists were not any more important than each panelist's reputation for reputable and rigorous application of the scientific method to social issues.

The work of the panel was of three parts. First, the panel was to meet to discuss the state of the research literature on desegregation and draw up a list of criteria for methodologically excellent studies in this field. Secondly, the criteria were to be applied to the body of desegregation research in order to screen out methodologically deficient studies. Third, each panelist was to arrive at a conclusion about the effects of desegregation on black achievement from his own meta-analysis of the remaining studies. The panel was then to analyze the differences amont the individual conclusions. It was thought that such an attempt at individual analysis within the dicipline of a group--with its built-in requirements of discussion, proofs, criticism, and rebuttal--would constitute a significant improvement over previous attempts to draw conclusions from the research literature.

The panel decided upon a list of twenty-seven "negative" criteria, any one of which would eliminate a study from consideration. Studies could be eliminated for, among the twenty-seven reasons, having no control data, being non-empirical, using an unknown sampling procedure, using dissimilar tests in pre-tests and post-tests, involving groups that were initially non-comparable, and presenting no pre-test means or post-test means. Atotal of 157 studies was identified, and when the criteria were applied to these studies, only nineteen studies remained--an astonishing result considering that the most rigorous effort to date, Wortman's, had included thirty-one studies and that Krol had used fity-five studies while Crain and Mahard had used ninety-three.

Individual and collective

analyses

Individual analyses of the nineteen studies of the analyses by Cook have now been published by the National Institute of Education in a volume entitled School Desegregation and Black Achievement. The volume reveals a remarkable convergence about the fundamental question.

Armor decided that "the conclusion is inescapable: the very best studies available demonstrate no significant and consistent effects of desegregation on black achievement." Walberg concluded that "school desegregation does not appear to prove promising in the size or consistency of its effect on learning of black students." Stephan decided that "these results appear to indicate that verbal achievement improves somewhat, but math achievement shows little effect as a result of desegregation studies that met the NIE minimal criteria show some moderateacademic benefit to black children when they attend desegregated schools. . . . [T]he magnitude of these effects translates into rather trivial increase of about twenty points on the typical SAT." Wortman found a "two-month gain or benefit for desegregated students," Cook decided that all the analyses taken together justified four conclusions: "(1) desegregation does not decrease the achievement of black children; (2) it probably does not increase math achievement; (3) it probably raises reading scores; and (4) the increase in reading scores is somewhere between .06 and .16 standard deviation units or about two and six weeks." In his own paper, Crain came up with an effects size but included almost no commentary about it. Instead, he used almost his entire paper to argue that the selection methodology of the panel was inferior to the methodology employed by Mahard and himself.

Cook added that he was "impressed by the degree of correspondence between the panelists when only 19 core studies are considered. None achieves negative estimates; all achieve larger estimates from reading than math, and the largest single difference--between Armor and Miller for reading gains--is of a magnitude many would consider small--viz., a difference of about one month of gain." Also looking at the work of the panel as a whole, Miller said that the "comparisons of mu effect size computations with those of Armor, Stephan, and Wortman for each study reveal that they agree fairly well." And Armor agreed, saying that "the range of differences is small in comparison with previous debates on this issue." Furthermore, Walberg observed that his analysis, Krol's, and Crain's prior analysis with Mahard were "in surprisingly close agreement considering the widely different selection criteria and number of studies in the three syntheses."

Zeroing in on each author's main conclusion does not do justice to the factors and qualifications each author considered in arriving at his conclusion, nor does it permit inclusion of the author's evaluations of their conclusions. space does not permit an elaborate discussion of every topic brought up by these authors. But some deserve brief mention.

Evaluation and conclusions

Cook and Stephan pointed out that in the area of reading, desegregation was associated with negative effects on black student achievement in programs of mandatory desegregation (four studies) but generally positive effects in programs of voluntary desegregation (fifteen studies). Armor and Miller observed that when rigorous studies show positive effects, it is likely that such gains are not due to racial mixing but rather to any changes in educational programs that may have occured simultaneously with disegregation. Additionally, Walberg's paper was a kind of elaboration of this observation by Armor and Miller since most of it was a devoted to an analysis of what research has shown are more efficacious ways of raising achievement than desegregation. Moreover, both Miller and Stephan included remarks about what past research has shown might be the proper conditions for black achievement. Stephan pointed out, among other things, that blacks in desegregated schools may have more anxiety with regard to achievement because of negative comparisons of themselves with white students; and Miller pointed out, among other things, that while white students seem to accept black students who are academically equal, white acceptance does not cause black academic achievement.

Cook, Armor, Wortman and Walberg all commented on and criticized the validity of the Crain and Mahard contention that desegregation can have a significant effect on black student achievement provided that it begins at the first grade or before. The four agreed that many of the studies upon which Crain and Mahard based this contention had serious methodological flaws, the most important of which usually had to do with the fact that the pre-tests were given to different groups and the fact that pre-and post-tests measured different things, thus making genuine comparisons of the test results impossible. Both Armor and Cook pointed out that when the methodologically-weak studies were eliminated from Crain and Mahard's analysis, their conclusions were roughly equivalent to the conclusions of the panel.

No members of the panel concluded that the small gains found--whether measured in months or points on the SAT--would accumulate annually over the school life of a student. Both Miller and Armor remarked that these and other studies seemed to indicate that the initial gains from desegregation were likely to be the only gains. Miller conducted an additional analysis of the three studies from this group of nineteen that permitted a comparison of the gains of black students to the gains of white students. He found that white students gained more than black students and that, therefore, desegregation seemed to ease an increase in the achievement gap between white and black students. On this point, Harold Gerard has concluded from his prior joint study with Miller that the gap between white and black achievement widened as students moved from one grade to another.

Miller and Wortman maintained that, across the entire field of desegregation research, studies with weaker designs tended to show more positive effects. miller also mentioned that Krol had figured that weaker studies tended to show positive effects twice as large as the effects shown in stronger studies. Such a correlation between lack of rigor and the finding of positive effects may point to a bias among the group of researchers in this field. On the other hand, Crain argued that technically better research designs were correlated with more positive effects, but he reached this conclusion from analyzing twenty studies selected from his list of ninety-three according to his own notion of the "best" criteria.

What effect has desegregation on black student achievement?

To summarize the NIE approach: Seven leading research scholars were commissioned to investigate the same field of social science research at the same time, usingthe same core group of studies as the basis of analysis. The core group of studies was selected according to a list of rigorous criteria developed and agreed upon by the researchers themselves in group cosultation. Each scholar was asked to conduct his own objective analysis of the studies approved by te criteria. Each had the opportunity to comment on every other paper, and one of the seven was required to analyze both the conclsions and methodologies o the other six. The approach as a whole, with its intense focus, its multiplicity of initial viewpoints and lack of ideological purpose, its element of checks and balances, its opportunity for direct criticism and cross-fertilization of ideas, its complte freedom of speech coupled with an emphasis on scientific analysis, and its provision of a common ground as an organizing princple of the discussion, produced the most comprehensive and in-depth treatment of the issue ever and, very likely, a definitive conclusion: desegregation has small positive effects on black student achievement in reading and no effects on black achevement in mathematics. Additionally, the panel's papers clarfied the results of previous research in the field and demonstrated that the panel's conclusion was in substantial agreement with that research.

The National Institutes of Health apparently use the panel approach for the same purposes: when the meaning of research in a field is in dispute and a consensus or generalized notion of the "state of the research" is thought necessary. The approach could also be used to clarify disputed research in other areas of education, among which are compensatory education, bilingual education, reading instruction, the effects of class size, and effective schools. Additionally, the approach could be efficacious in shedding light on other controversial issues of social policy, as for example, drug abuse or pornography.

Thirty years after Brown, the issue of academic achievement continues to occupy center stage in school desegregation suits. In the 1982 case of Kelly v. Metropolitan County Board of Education in Nashville and Davison County, Tennessee, the Sixth Circuit Court of Appeals, citing the work of Crain and Mahard, overturned the finding of the district court judge that desegregation had only slightly improved the blck-white achievement gap and concluded that "the impact of these studies is tha desegregation raises the level of black achievement. In the very recent (July 9, 1984) case of Riddick v. School Board of the City of Norfolk, a Federal district court heard "a significant amount of expert testimony ... on the question of whether the scholastic achievement of students, black and white, is affected when the school they attend is racially mixed." But the judge decided not to attempt "to balance the experts" and came to no conclusion about the effects of school desegregation on achievement.

Appendix

Mention should be made of the long sections of the papers of both Miller and Stephan dealing with aspects of desegregation other than academic achievement. Miller, remarking that "much past theorizing has not withstood the test of data," observed that the theory that segregation lowered the self-esteem of black children--the theory accepted by the Supreme Court in Brown--"has not withstood empirical analysis." Furthermore, he cautioned that desegregation's effect on race relations is problematical since "resegregation is common in desegregated classrooms." With regard to the same two issues, Stephan concluded that "it appears that the social scientists who participated in Brown used an invalid assumption as a basis for their argument that desegregation would increase the self-esteem of Black students" and that "desegregation rarely occurs under conditions that would lead to improvements in race relations."

COPYRIGHT 1984 U.S. Government Printing Office
COPYRIGHT 2004 Gale Group