Effectiveness of Challenge-Based Instruction in Biomechanics

Journal of Engineering Education, Oct 2006 by Roselli, Robert J, Brophy, Sean P

Each year we compared students' performance on specific knowledge-based questions on the final exam, and on their perceptions of the course using a variety of surveys and questionnaires. Based on these outcomes we made revisions to the modules before implementing them again. Changes were also made to die way materials were presented in the traditional course in response to student performance on examinations. Therefore, neither the traditional or CBI course remained completely static. Efforts were being made to continually improve both classes over time.

D. Instructors

For a variety of reasons, it was not possible to use the same instructor in both the traditional and challenge-based courses. Therefore, two different instructors were used in this study. The senior instructor, and first author, taught the CBI sections and a more junior faculty member witli research interests in biomechanics taught the traditional sections. We controlled for instructor bias by comparing students' course evaluation results from the years the treatment instructor taught the course in the "traditional" method of instruction with the performance of the control instructor during the years of the comparison study.

E. Course Content

Although we were unable to control for the teaching experience, research interests, age or enthusiasm of the instructors, we were able to carefully control the taxonomy that was presented in the course. The control instructor was provided with all of the materials (lecture notes, homework questions, exam questions) used over the years in the course by the treatment instructor, and closely followed the "traditional" syllabus that was already in place. In later years, the control instructor began to use some of the materials developed by the treatment instructor in the challenge-based course, including in-class formative assessment and technology tools, such as the Free Body Diagram Assistant [27] for out of class assistance. We anticipated that these activities might diminish the difference between the two courses. However, these HPL activities and techniques did not involve challenge-based modules. The major distinction between the two methods of instruction, then, was that one instructor used challenge-based instruction and one instructor did not. Observations made in both classes, discussed below, were made to help quantify in-class differences in HPL activities.

F. Instruments and Measures

A combination of classroom observations, surveys, and exam questions, were used to measure differences in classroom instruction, students' perceptions of instructional methods, and students' knowledge of biomechanics

1) Classroom Observations: We used classroom observations to assess the fraction of class time in CBI and traditional classrooms spent in HPL activities. External observers from Vanderbilt's Peabody College used the VaNTH Observational System (VOS) to record the degree of interaction during class between instructors and students. The VaNTH Observation System (VOS) [28] estimates the percentage of class time spent in activities that are knowledge-centered, learner-centered, classroom community-centered, and assessment-centered. The VOS instrument codes a sequence of events that capture who is talking to whom, what they are talking about (academic knowledge, behavior modification, setup etc.) and what technology is being used. A total of 19 CBI classes and 14 traditional classes were observed during the last five semesters of this study. The VOS was not available during the first semester of the study. Cksses were selected by the observers for content being taught and the relationship to test time (e.g., not a class just before an exam that might be more of a review session). These measurements represented more than 2,100 minutes of class time and over 8,000 individual observations (approximately one observation every 15 seconds). The average of each HPL activity over all classes was computed for traditional and CBI modes of instruction, and means were compared using two-tailed Student t-tests, assuming unequal variance.