Assessing Pedagogy in Bioengineering Classrooms: Quantifying Elements of the "How People Learn" Model Using the VaNTH Observation System (VOS)

Journal of Engineering Education, Oct 2008 by Cox, Monica F, Cordray, David S

ABSTRACT

One goal of the VaNTH Engineering Research Center is to estimate the effects of the "value added" to bioengineering student learning as a result of "How People Learn" (HPL) framework interventions. A necessary step in that process is to assess pedagogical differences in both lecture-based and HPL-oriented courses. Data from 28 bioengineering courses, over five semesters, were analyzed using a newly developed HPL Index. This index, developed from the Classroom Interaction Observation portion of the VaNTH Observation System, reports levels of HPL-inspired pedagogy, traditional pedagogy, and classroom organization within a class using codes for different types of faculty and student interactions assigned by an observer in real time. Results confirm the HPL Index's ability to distinguish pedagogical practices based on HPL principles and pedagogical practices based on traditional, non-HPL pedagogy.

Keywords: classroom assessment, engineering pedagogy, observation systems

I. INTRODUTION

As the demand for innovative technology increases within the United States, the number of highly qualified engineers must also increase. ABET requires that engineering graduates demonstrate several skills, such as the ability to solve engineering problems, to apply science-based, engineering, and mathematics knowledge, and to effectively work in interdisciplinary teams (Herkert, 1999). These requirements affect over 2,700 engineering programs at approximately 550 colleges and universities nationwide.

In an effort to address these requirements within engineering, researchers have begun conducting empirical studies to identify pedagogical practices intended to optimize these desired skills in the next generation of engineers. Teaching and learning within engineering differs from teaching and learning within other disciplines because engineering is a "hard/applied" field of study, focusing on inquiry strategies, mastery of the environment, and the development and application of products and processes (Lodahl and Gordon, 1972; Biglan, 1973; Neumann, Perry and Becher, 2002). In-class student collaboration within engineering classrooms increases students' critical thinking skills and has positive effects on student achievement, persistence, and attitudes compared to lecture-based engineering classroom environments (Cooper and Robinson, 1998; Cudd and Wasser, 1999; Springer, Stanne, and Donovan, 1998). In addition, engineering classes with greater emphases on faculty-student interactions and faculty guidance have positive effects on student learning (Cabrera, Colbeck, and Terenzini, 2001).

To meet the educational standards of ABET and to increase the quality of the students' educational experience within engineering classrooms, valid assessments of current engineering classrooms are needed. One such discipline-specific instrument, the VaNTH Observation System (VOS), was developed to capture pedagogical practices in bioengineering classrooms (Harris and Cox, 2003). It has been used to observe over thirty bioengineering courses at two research universities. Although the VOS captures the nature and quality of faculty and student interactions within engineering courses, its use to date has not been optimized because of the absence of an algorithm for combining the distinct observation segments into a coherent index that can be used to register the quality of pedagogy exhibited in courses.

This paper describes the development and application of a newly developed index that parses data collected from one portion of the VOS into categories representing traditional and nontraditional instruction and distinguishes pedagogical style from other activities associated with classroom organization or management. Overviews of VOS studies are described, and the criterion contrast of the index is examined. Differences between traditional and HPL-oriented courses within one biomedical engineering program (at Vanderbilt University) are explored and presented, and connections to practice are discussed. The next section provides an overview of the VaNTH Engineering Research Center (ERC).

A. The VaNTH ERC Model of Learning and Instruction

The VaNTH Engineering Research Center (ERC) for Bioengineering Educational Technologies was established in 1999 with funding from the National Science Foundation (NSF). VaNTH is a multi-university ERC developed to maximize the educational experiences of bioengineering students at Vanderbilt University, Northwestern University, the University of Texas at Austin, and the //arvard/Massachusetts Institute of Technology Division of Health Science and Technology. VaNTH involves a collaboration of professionals from Bioengineering Domains (e.g., Biomechanics and Biotechnology), Learning Sciences, Assessment and Evaluation, and Learning Technology. The VaNTH ERC is the only Center funded by NSF devoted solely to bioengineering educational technologies. The goal of the VaNTH ERC is to "unite educators and engineers, in industry and academia, to develop curricula and technologies that will educate future generations of bioengineers (VaNTH, 2003). These curricular changes were guided by the "How People Learn" (HPL) framework (Bransford, Brown, and Cocking, 1999), a synthesis of research on learning that complements other pedagogical practices (e.g., constructivist, problem-based learning) that have been found to be effective within many classroom environments.