Effectiveness of Challenge-Based Instruction in Biomechanics

Journal of Engineering Education, Oct 2006 by Roselli, Robert J, Brophy, Sean P

2) Surveys: Student surveys in CBI classes were used to elicit students' perceptions of how well the course was informed by the HPL framework and their reactions to various methods used during the semester. The "Experiences in and Benefits from This Course" (EBC) survey was constructed from several well established surveys [29]. Items that captured the features of the HPL Framework were selected. Students in both sections of the course completed the survey at the end of the semester. For example, to evaluate learner centeredness, students were asked to rate statements like "Learning connects with prior knowledge," "Instructor relates course to real life," and "Teaching is tailored to student aptitudes" using a 5 point scale (1 = never or almost never to 5 = always or almost). To evaluate assessment centeredness, students rated statements like: "Instructor assesses our preconceptions," "Instructor assesses what needs clarification," "Instructor checks student understanding," and "Instructor helps students answer their own questions"

A Student Exit Survey was used to capture students' reaction to various instructional methods used in the course (e.g., use of challenges, classroom communication system, in-class group problem solving, generating ideas). These questions targeted our theories and assumptions about why challenge-based instruction might be beneficial. Therefore, these questions provide a link between students' perceptions of the instruction and our anticipated benefits of the instructional method.

Finally, the results from end of course evaluation surveys used by the School of Engineering were used to compare equivalence of instructors as an initial control and to measure post-CBI changes of the challenge-based instructor. This survey provides historical information on the instructors which we can use as a benchmark for change in instructors' teaching using the learners' perceptions as an indicator.

3) Knowledge-Based Questions: Twelve to twenty representative knowledge-based questions were selected for comparison by the instructors of BME 101 each fall. All comparisons were made between identical questions asked on the final examination. Final exams are not returned to students, so students did not have access to previous final examination questions. Each instructor designed questions that were shared with the other instructor. Therefore, the number of core questions asked and compared increased over the years. A total of 50 comparisons between knowledge-based questions were made during this study.

A scoring rubric was developed for each question by the two instructors. Performance on each question was compared between the control and CBI classes by applying a one-tailed Student t-test, assuming unequal variance. All problems in both courses were graded according to the scoring rubric by a graduate student with expertise in biomechanics. A different student was used each year. It is unlikely that different graders would interpret the rubrics differently, but even if they did, we did not compare scores from year to year. The Fall/Spring knowledge-based comparisons each year were based on identical questions graded by the same grader. Student names were removed from exam papers and the purpose of the study was not revealed to the graders. Effect size (ES) was computed as the difference in mean score between the CBI and traditional groups divided by the pooled standard deviation for knowledge-based questions. Questions were divided into two groups: more difficult and less difficult, depending upon whether the average of all student scores (both CBI and Control) on that question was less than or greater than the average of all student scores for all questions. A paired Student t-test was used to compare control and CBI scores for all questions, the less difficult questions and the more difficult questions. Means were considered to be significantly different if p