Testing computer based high school science modules in a classroom setting

Journal of Geoscience Education, May 2003 by Bartholomew, James W Jr, Carpenter, John R, Owens, Thomas J

Figure 2 shows the mean and standard deviation for the combined data set. Figure 3 shows the results of the paired sample t-test. The result was t = -10.24, significant with alpha = 0.05 for a 95% confidence interval. This test shows that on average student scores on the post-test will be between 5 and 7 points higher on a test of 33 questions over the course of all four modules.

It should be noted that the student scores increased 5 to 7 points out of 33 (15% to 20%) after the students had completed nearly a year of instruction in related material. It might have been expected that we would have seen a "ceiling effect" (Carpenter, 1983) in the pre-test scores such that gain scores would show no change but this was not the case. Other factors that potentially could have negatively influenced this outcome were the large drop in the number of students due to end-of-year activities, not to mention a serious drop in the motivation of seniors already accepted into the college of their choice. This is an important point. According to the cooperating teacher, the senior class was pretty much "going through the motions" without any consistent effort on their part. The fact that a significant difference between pre- and post- tests scores occurred despite these difficulties makes this result even more meaningful. Therefore, we believe that the gain in scores can be taken as evidence that students can learn science material using the SCEPP modules in general.

It is possible that other factors influenced the student gain scores. We will likely try to determine if affective factors such as the novelty of using the new software and seismic data might have been an influence. One new study will attempt to determine if student interest might have been enhanced by the use of real scientific equipment and data, and if so, if the enhanced interest might have had an effect on subject matter gains. We will attempt to use a questionnaire consisting of several Likert scale questions to assess student attitudes. Also trying the modules in a 9th grade physical science class may yield very different results with students who have not received such a large amount of instruction in physical science. Finally, going to one post-test and one pre-test test when testing the modules will help prevent students from becoming bored with taking multiple tests. The project goal that this study could not meet was to determine the individual module effectiveness as a part of regular classroom instruction. If we are able to obtain a large enough group of participating teachers, we will have the luxury of testing just one or two modules at a time and still test all the modules more than once. Unfortunately, without the use of a control group, it will be difficult to answer how well the modules perform when integrated as a part of instruction and not done after instruction has taken place. Separating the effect of instruction from that of the module itself will be impossible without the use of a control group. Again, requesting a few teachers to test only one or two modules in order to reduce the time needed for a control group could enable a new experimental design to be used. All further testing will be done with approximately twice the number of questions for each module pre/post-tests in order to increase reliability for the tests.