Using comparison groups in school counseling research: a primer

Professional School Counseling, June, 2006 by Sheri Bauman

This article describes comparison group research designs and discusses how such designs can be used in school counseling research to demonstrate the effectiveness of school counselors and school counseling interventions. The article includes a review of internal and external validity constructs as they relate to this approach to research. Examples of relevant research using this design are presented.

*********

The lack of a sound research base in the field of school counseling has been lamented for many years (Allen, 1992; Bauman, 2004; Cramer, Herr, Morris, & Frantz, 1970; Lee & Workman, 1992; Loesch, 1988; Whiston & Sexton, 1998; Wilson, 1985). The recent emphasis on research in No Child Left Behind legislation (2002) and the ASCA National Model[R] (American School Counselor Association, 2005) has moved the need for rigorous empirical research to the forefront. The ASCA National Model stresses that school counseling programs include learning objectives that are based on measurable student outcomes and that are data-driven and accountable for student outcomes. The focus on data and measurement makes clear that school counselors can no longer avoid conducting research and using empirical research to make decisions.

The nature and goals of such research are the subject of a recent debate. Brown and Trusty (2005) have contended that research should focus on demonstrating that well-designed and appropriate interventions used by school counselors are effective, and they further argued that research investigating whether comprehensive school counseling programs increase student academic achievement is not productive given the presence of numerous confounding influences. Sink (2005) disagreed, noting that school counselors are expected to contribute to the total educational effort to raise academic achievement. He advised that research to examine how school counselors influence achievement can be conducted using carefully selected methodologies, and while not definitively establishing causality, such research can provide strong evidence of the impact of comprehensive school counseling programs on student achievement.

In their review of school counseling outcome research from 1988 to 1995, Whiston and Sexton (1998) found that of the 50 published studies they located, most provided only descriptive data, used convenience samples, lacked control or comparison groups, used outcome measures of questionable reliability and validity, and did not monitor adherence to intervention protocol. Such studies do little to add to the knowledge base of the profession, and they do not meet established standards for scientific rigor.

In an era of limited resources for education and "accountability" becoming a watchword, counselors must demonstrate how they contribute to the academic success of students. Heartfelt letters of appreciation and positive comments by constituents, while sincere, will not convince stakeholders and holders of purse strings of the value of the profession. School counselors, occupied by providing services in schools, often neglect to demonstrate their importance until their positions are considered for reduction. This reactive approach is less likely to sway opinion than ongoing proactive efforts to use research effectively. Collecting, analyzing, and discriminating data that provide evidence of counselors' effectiveness are consistent with the professional goals and models that define the profession.

Under No Child Left Behind, school counselors (along with other education professionals) are called upon to demonstrate their effectiveness using quantitative data such as evidence of academic achievement, attendance and graduation rates, and measures of school safety (McGannon, Carey, & Dimmitt, 2005). No Child Left Behind and the ASCA National Model both emphasize the importance of scientific, rigorous, well-designed research as an essential component of modern school counseling programs. These guidelines indicate that conducting research is no longer a peripheral activity that a few counselors might attempt but is a central part of the role of all school counselors. The ASCA National Model says the following about data:

   Data analysis: Counselors analyze student
   achievement and counseling-program-related
   data to evaluate the counseling program, conduct
   research on activity outcomes and discover
   gaps that exist between different groups
   of students that need to be addressed. Data
   analysis also aids in the continued development
   and updating of the school counseling
   program and resources. School counselors
   share data and their interpretation with staff
   and administration to ensure each student has
   the opportunity to receive an optimal education.
   (ASCA, 2005, p. 44)

Although most school counselors have had a graduate course in research methods (and perhaps statistics), these introductory courses typically are designed to prepare students to be critical readers of research. Even among those few students who conduct research in their graduate training programs, it is the rare school counselor who continues to do research once he or she is a practicing school counselor. Responsibility for the absence of research from school counselors' job description lies not only with the counselors, but also with the administrators and district officials who do not require or value research. No Child Left Behind has raised the awareness of educators in all fields that accountability is expected; data are the foundation for educational decisions, including decisions about counselors. In this climate, there is greater support (some might say pressure) for research.