Searching for the Difference: A Controlled Test of Just-in-Time Teaching for Large-Enrollment Introductory Geology Courses

Journal of Geoscience Education, Jan 2006 by Linneman, Scott, Plake, Terri

ABSTRACT

At Western Washington University (WWU), we tested the effectiveness of Just-in-Time Teaching (JiTT) methods in introductory geoscience courses. JiTT is an interactive-engagement teaching method developed for introductory physics courses (Novak et al., 1999). The key to the JiTT method is a short time frame feedback loop in which student learning informs instruction. The JiTT method includes several distinct components. (1) Web-based WarmUp exercises check on students' comprehension of reading assignments and ability to connect new concepts to those previously developed. Students submit answers to these exercises via the web up to four hours before each lecture. (2) Interactive lectures based on WarmUp responses emphasize problem-solving skills. (3) Web-based puzzles wrap up topics with critical thinking problems. We conducted our test using two sections of Geology 101 at WWU: the experimental section applied JiTT methods and the control section was taught using the traditional lecture format. Our experimental and control sections were non-major general education courses, had the same learning objectives, and objectives were assessed using the same exams. Students completed a First Day Survey providing baseline data about their demographics, science courses taken previously in high school and college, their attitude about studying science and their knowledge about basic Earth science concepts. On the first exam, the JiTT students scored significantly higher than the students in the traditional lecture section by an average of 2 points out of 50. The JiTT students' combined average on three exams during the course was higher, but not statistically significant. When the data from the First Day Survey were taken into account using multi-variable regression analysis, all differences in performance on exams becomes insignificant. We interpret our results to indicate that scores on objective (emphasizing knowledge and comprehension) exams may depend strongly on students' incoming prior knowledge, more than on the specific methods used to teach the class. Student evaluations of the course taught using JiTT methods were very positive toward the JiTT methods and their experience with introductory geology. Although we hope that such affective differences foreshadow long-term enhancement in learning (that our study did not assess), we caution instructors not to rely only on exam scores or student evaluations to gauge teaching effectiveness. Conversely, we recognize that scores on objective exams may not capture development of higher order thinking that the JiTT method may foster.

INTRODUCTION

Most recent reports on improving introductory college science education recommend increasing interactivity between students and the instructor and amongst the students themselves. Scores of superb teaching strategies aimed at increasing interactivity via writing, speaking, and group exercises have been published in the pages of this journal (e.g. MacDonald et al., 1992; Schweitzer, 1995; Tewksbury, 1995a, 1995b). Instructors at small colleges with relatively low course enrollments (n

This paper describes a study of the effectiveness of Just-in Time Teaching (JiTT) in Geology 101, an introductory geology course at Western Washington University (WWU). JiTT is a technology-based interactive strategy designed by physics educators (Novak et al., 1999) that draws its practical elements from industrial Just-in-Time methods using high-speed communications and rapid distribution systems to improve efficiency and flexibility. The JiTT philosophy is essentially constructivist as it uses web-based resources to uncover student misconceptions and inform the planning of classroom activities. The 21st century novelty of the approach is the short timeframe (minutes to hours) on which this feedback loop occurs every class period. Grove (2002) employed a strategy of regular online homework (Virtual Voyages) as a means of more actively engaging students with the course material in a large-enrollment general education ocean science course.

WHAT IS JITT?

In a JiTT class, students are required to respond electronically to precisely designed web-based assignments (described below) that are due a few hours before class. The instructor reads the student submissions "just-in-time" to adapt the classroom activities to match the students' needs. The pedagogical core of JiTT is the 'feedback loop' formed by the students' outside-of-class preparation that fundamentally affects what happens during the subsequent in-class time together. JiTT's creators claim, "the students come to class prepared and already engaged with the material, and the faculty member already knows exactly where the students are and where classroom time together can be best spent." (Novak et al., 1999, p. 3). Significantly, the feedback cycle is repeated several times each week, a strategy designed to compel students to extend their study times and avoid pre-exam "cramming".

The web-based JiTT components of Novak et al. (1999) consist of WarmUps, Puzzles and Good-For Essays. The WarmUp exercises consist of three questions: 1) a short essay on the topic for the day; 2) an estimation question; and 3) a multiple-choice question. The WarmUp is due a few hours before the scheduled class meeting to give the instructor time to read through the answers, looking for patterns of comprehension (or miscomprehension). Several representative answers taken directly from student responses are incorporated into classroom discussion at appropriate times. WarmUps are scored on the basis of completeness and effort, not ultimate correctness of their answers. The Puzzles are weekly, single question Web assignments that prompt classroom discussion as a review of the material of the preceding week. These assignments emphasize problem solving: usually they are a scenario with some kind of twist or apparent contradiction. They are designed to challenge commonly held misconceptions and should present a significant challenge to the beginning student. Good-For Essays are infrequent short writing assignments that ask students to reflect on the relevance of course topics to their daily lives (Novak et al., 1999).