Real-Time Analysis of Student Comprehension: An Assessment of Electronic Student Response Technology in an Introductory Earth Science Course

Journal of Geoscience Education, Sep 2004 by Greer, Lisa, Heaney, Peter J

ABSTRACT

Electronic student response technologies (SRT) are capable of assessing teaching and learning methods in real time, and they offer an exceptional means of introducing active learning protocols in classes with large enrollments. These wireless systems allow students to key in responses with remote control units to questions posed by an instructor in the classroom. Student responses then are displayed in real time, allowing both students and instructors to gauge student comprehension instantaneously. From Spring 2002 to Spring 2003, we utilized SRT in 4 sections of a high-enrollment introductory Earth Science course (Geosc 020: Planet Earth) at Penn State University. We conducted a multi-faceted assessment of the use of SRT in our course that included quantitative and qualitative perception data from students enrolled in the course and faculty/administrator visitors to our classroom. Our preliminary assessment of the pedagogical merits of SRT in our course suggests that this technology is an effective tool for introductory geoscience education.

INTRODUCTION

A number of studies have shown that traditional methods for teaching science to non-science majors are limited in their effectiveness (Pinet, 1995; Dufresne et al., 1996; Mazur, 1997; DeCaprariis, 1997; Gerace et al, 1999; McManus, D.A., 2002; McConnell et al, 2003). In th� conventional approach, instructors present information in a lecture format and the students passively gather that information by listening and taking notes. Several newer teaching models stress active student participation, in which students process material presented in class via discussion with the instructor and peers, problem solving, and group activities interspersed with periods of listening and note-taking (Angelo and Cross, 1993; Bykerk-Kauffman, 1995; Mazur, 1997; Yuretich et al., 2001; Crouch and Mazur, 2001). However, even when more progressive teaching methods are employed, it often is difficult to gauge student involvement, interest, and level of comprehension.

Electronic student response systems present one method for overcoming these barriers to engagement and effective learning (Dufresne et al., 1996; Mestre et al., 1997; Wenk et al., 1997; Gerace et al., 1999; Judson and Sawada, 2002; Meltzer and Manivannan, 2002; McConnell et al., 2003). When an instructor can query and collect responses from every individual in his or her classroom, the instructor can gauge instantaneously what students understand, and more important, which concepts they are failing to grasp. This feedback allows the teacher to spend less time on material that already has been processed in favor of focusing on 'problem areas'.

Inspired by our on-site observation of classroom response systems used in physics and biology courses at the University of Massachusetts at Amherst in 2001, we decided to test similar technologies in Geosc 020: Planet Earth, a general education course at Penn State with mean semester enrollments of -145 students per section (Table 1). In Spring 2002 we initiated our experimentation with a commercially manufactured electronic response technology produced by elnstruction called the Classroom Performance System (CPS). While response systems have been successful instruments of active learning in university-level physics, chemistry, and biology courses for well over a decade (Littauer, 1972; Dufresne et al., 1996; Mestre et al., 1997; Gerace et al, 1999; Judson and Sawada, 2002; Meltzer and Manivannan, 2002), SRT use in the collegiate Earth Sciences is in the early stages of implementation (McConnell et al., 2003).

The primary pedagogical goal of our SRT integration project was the creation of an active-learning environment in a large introductory Geoscience course (Geosc 020), which previously had been taught in traditional lecture format. Specific objectives of the project were to encourage active student participation in Geosc 020, to enhance student communication and collaboration skills, to develop problem solving skills during class time, and to increase student attendance in class from the historic mean attendance of -50% by mid semester.

As a necessary component of this effort, we wished to evaluate the perceived effectiveness of SRT technology in enhancing both student learning and instructor teaching. To address this requirement, we conducted a multi-faceted assessment of student opinions on the benefits and disadvantages of using this technology in our class. We compiled quantitative data via extensive student surveys and student attendance data. We collected qualitative data from oral student surveys and from interviews with faculty peers who observed our operation of SRT in our classes (Greer et al., 2002; Heaney and Greer, 2003).

METHODS

What is the Classroom Performance System? Student response technology has been in development for several years. The ClassTalk system (Dufresne et al., 1996; Webking, 1998; Mestre, pers. comm., 2001), pioneered primarily by the physics faculty at the University of Massachusetts at Amherst, is a hard-wired response system with software that allows an instructor to view a real-time 'map' of the classroom. Each student response is color-coded by seat and student name or identification number. Unfortunately, installation of hard-wired electronic response systems is prohibitively expensive for many universities.