Early Alerts II: An Experimental Evaluation

Research & Teaching in Developmental Education, Fall 2003 by Brothen, Thomas, Wambach, Cathrine, Madyun, Na'im

Abstract

This study builds on our previous research that found no beneficial effect for sending early alerts to low performing students in our general psychology class. We used a randomized control group design to study the effects of early alerts that had a high probability of being read by students and acted upon by their advisors. We examined the effects of early alerts on four measures of course performance: students' point totals at the end of 5 and 10 weeks in the semester, final point totals, and the amount of time students worked after the alert was sent. Our results are consistent with prior research suggesting that early alerts have little impact on student performance. We discuss the potential benefit of early alerts even though they are only marginally effective.

Recently (Hansen, Brothen, & Wambach, 2002) we reported on a study of the use of early alerts to improve underprepared students' performance in our general psychology course. Early alerts are components of "intrusive academic advising" systems that are designed to track student academic progress closely and identify where assistance is needed early in the term so as to improve performance and retention (Garing, 1993). Early alerts referring students to advising and other college services, which are sent to students not making adequate academic progress in classes, are crucial to such programs. This intervention is based on the assumption that students may be unaware of their status in the course or may not know where to go for assistance with problems. And, as the name suggests, the assumption also is that this should be done in the first 2 or 3 weeks of the course instead of waiting until it is too late for students to get on track. By providing feedback on performance and information on resources to students, colleges hope to improve retention and student success.

A variety of methods have been used to deliver information to students and advisors about problems with performance. McKinney (1997) reviewed several of them and found that some institutions use paper and pencil forms and send written letters to students while others use electronic forms of communication. Decisions about how to implement the process are likely driven by the resources the institution has to spend and beliefs about the most effective ways to contact students.

In our general psychology course, we have many reasons to attempt to reach students who are not completing work on schedule. The class enrolls about 240 students each semester, and includes students with diverse academic preparation. To accommodate the broad range of learners we enroll, we use Keller's (1968) personalized system of instruction (PSI) that we have augmented with written and computer based activities (Brothen & Wambach, 2000). Students read the text, complete the study guide, and then use a variety of computerized practice exercises to get feedback on their mastery of the material. When they are ready, they take chapter quizzes. Students can complete the practice exercises in our computer classroom, or over the Internet from anywhere they have access to a computer. The chapter quizzes must be completed in the classroom, which gives us an opportunity to work with students individually as they learn psychology. Because students must complete quizzes for 18 chapters, they must begin completing work early in the term. We have immediate information about which students are and are not engaging the course tasks.

There are many advantages to this pedagogy, especially for underprepared students (Brothen & Wambach, 2000). Students get a great deal of feedback and as much personal attention as they want. The flexibility of the structure accommodates a wide variety of personal situations. However, it does open possibilities for some students to postpone their work, and for test anxious students to avoid quizzes. This can become very problematic for students late in the semester when they are trying to catch up on the large volume of material that must be mastered in any general psychology course.

In addition to procrastination and test anxiety, there are other reasons why a student might be having trouble in a general psychology course, some of which can be exacerbated by the PSI method (Wambach & Brothen, 2001). Some students have learning disabilities that make reading the text and test items more challenging, and others have multiple personal problems that interfere with academic progress (Brothen & Wambach, 1999). When a student is attending class, but not performing well on exams, we work individually with that student and usually do not contact the student's academic advisor. Our use of early alerts is primarily to contact students who are falling behind in the scheduled coursework and not attending class

Similar to Rudmann's (1992) findings of no effect for early alerts, we found no evidence in our Hansen et al. (2002) data that the alerts produced any obvious effect on student work or course performance. The early alerts did not seem to produce a turn around in student behaviors or contribute to ultimate success in the course for the students who received them. We suggested that student habits or personal problems might have much more effect on their academic progress than intervention strategies such as early alerts. Overall, we found no evidence that the alerts had an independent and useful effect on students' success in our course. However, it was possible that the alerts system in place at that time did not reliably deliver the information to advisors or students.