Evaluation of instructional design of computer-based teaching modules for a manufacturing processes laboratory

Journal of Engineering Education, Jul 2000 by Hailey, Christine E, Hailey, David E

ABSTRACT

Studies concerning student preferences and student learning as a function of the instructional design and delivery ofa computerbased teaching (CBT) module are presented. The studies were conducted in conjunction with the development of twenty-one CBT modules for an Introduction to Manufacturing Processes laboratory that emphasized metal removal. Study results indicate there is no statistically relevant difference in learning between students using material presented with traditional multimedia (35 mm slides and cassette tapes) and the identical material presented with digital multimedia Engineering students' preferences for interface design and audio-visual information presentation are also presented. The most important result is that learning outcomes of a reader-driven CBT module were found to be statistically lower than those associated with author-driven CBT module, especially for average and below-average students. These results suggest that if students must absolutely understand material, e.g., laboratory safety, the CBT should be author-driven. Based on these results, we speculate that average and below average engineering students are more linear learners. A hybrid scheme, where information presentation transitions from an author-driven to a reader-driven environment may help weaker students develop better non-linear, open-ended problem solving skills.

I. INTRODUCTION

Computer based teaching (CBT) programs are changing engineering education. Bengu and Swart' and Sears and Watkins2 have described learning modules developed for use on the World Wide Web (WWW). Cobourn and Lindauer,' Mosterman et al.,4 Harger,5 and Dobson et al.6 have described additional learning modules for a variety of engineering programs developed with various authoring software. The authors of References 3 through 6 distributed questionnaires to their students in order to assess student receptivity to the modules. All noted favorable responses from students.

Other investigators have developed CBT modules and assessment schemes to test student learning. Flori et al.7 developed a CBT module entitled "BEST" Dynamics with the goal of improving both teaching and learning. Students who interacted with the BEST software outside of class typically scored nearly a letter grade higher on a quiz when compared with students who largely did not use the software. Bailey et al.8 developed a CBT module on phase diagrams in a basic materials course. Roughly 40 students used the CBT module while 40 other students had a lecture-based presentation of the material. All students were then administered a multiple-choice quiz, and the results indicated that students who experienced the CBT material performed no better and no worse on the quiz when compared with the those who experienced a traditional lecture.

Wallace and Mutooni9 present a good review of other CBT projects, especially those developed for use on the WWW. The purpose of their study was to design a pedagogical framework for teaching via the WWW, implement a lecture in that framework, and then evaluate its effectiveness compared to a classroom lecture. They provided both "sequential and parallel reference access" to materials. Performance on a project was evaluated for students who attended a classroom-based lecture compared with students using the CBT module. The average performance of students who received web-based instruction was higher compared with those who received traditional classroom instruction. The students spent roughly as much time on the CBT module compared with time in lecture. The CBT students indicate$ they valued the ability to pace their learning. The authors do not comment on how the students accessed the material--sequentially compared with parallel.

The above investigations indicate there are not negative outcomes when CBT is used in place of or in conjunction with a traditional lecture. With this point in mind, we decided that the instructional design and delivery of CBT modules should be studied in more detail to determine what is most effective for engineering students. In this paper, we present the results of three studies concerning student preferences and student learning as a function of the instructional design and delivery of a CBT module. The studies are based on pre-laboratory learning modules developed for an introductory manufacturing processes course. A multidisciplinary team, composed of students and faculty from English and engineering faculty, developed these modules.

We focused our research on CBT modules for engineering students on three areas. First, we assessed comparative student learning with traditional multimedia (35 mm slides and cassette tapes) and digital multimedia. Second, we examined student preferences for interface design and audio and visual information presentation. Finally, we assessed student learning as a function of the instructional design of the CBT module.

II. BACKGROUND

Clark and others argue there is no compelling casual evidence that media or media attributes influence learning in any essential way.10 He argues that "design" technologies should draw on psychological and social-psychological research in order to select necessary information and objectives that provide an adequate learning environment that influences student achievement. This is not to be confused with Clark's term, "delivery" technologies, which he feels are necessary to provide efficient and timely access to the "design" methods and environments. We note that the investigators in Reference 9 claim to have found a difference in student performance as a function of the "delivery" technology. Hence, one topic we decided to investigation was an assessment of student learning as a function of traditional multimedia (mixed media in a non-digital format) when compared with digital multimedia.