Testing computer based high school science modules in a classroom setting

Journal of Geoscience Education, May 2003 by Bartholomew, James W Jr, Carpenter, John R, Owens, Thomas J

METHODS

This study reports on data gathered in connection with Objective 3, i.e. increase student knowledge of usual science concepts. For this study, we selected one school as a pilot site. The school was located in Pickens County near Greenville, South Carolina, and consisted of grades 9 through 12 with a of population approximately 3,000 students. The study had available approximately 110 students in four classes of honors physics and one class of "tech-prep" physics for students not intending to attend college but may want to attend a technical or vocational school. Classes consisted of primarily 12th grade students, with a small number of 11th grade students. The students were given four SCEPP teaching modules to use during the last 5 weeks of the school year. All the modules were taught at the same time to all classes consecutively. The experimental design used in this study was the One-Group pre-/post-test design of Campbell and Stanley (1963). Using this design, students of an experimental instruction treatment group (Experimental Group), are given both a pre-test on the subject to be covered by the treatment and an identical (or equivalent) post-test on the subject matter covered. In this design there is no Control Group of students, on whom a traditional treatment of the same subject matter is used. For our study, a control group was not used because it would double the number of class preparations the cooperating teacher would have to prepare and was considered unacceptable by the cooperating teacher in the study. With no control group, we could not compare the effects of the experimental approach and a more traditional approach on any gains in knowledge. Because of this, the modules were done after the students had already completed subject matter instruction and the school year was nearly over. This design is testing how much additional learning students accomplish using the modules. A set of four pre-tests, one for each module, was given prior to instruction. A post-test was given at the end of each module, before beginning the next modules instruction. Pre-test and post-test instruments were identical and consisted of 8 to 10 multiple choice questions for each module. Each test was taken using university scan sheets to allow the tests to be graded, item analysis to be done and a reliability coefficient to be calculated. The total number of test scores was 400. Item analysis would allow us to identify questions that were not "performing" appropriately. We could then correct or discard inappropriate questions in the next round of testing. The reliability coefficient was calculated to evaluate how well the tests were functioning overall and as a measure of how reliable our conclusions using data from the tests would be. Low reliability would cast doubt on any conclusions about our materials regardless of the scores or other statistical analysis.

RESULTS

The study suffered from a high degree of student absenteeism due to end-of-year activities, testing, etc., causing many students to miss part of a module, a pre-test or a post-test. The analyses, therefore, are based on a final number of students lower than the total number available.