Engineering online: Assessing innovative education

Journal of Engineering Education, Jul 2002 by Haag, Susan, Palais, Joseph C

Although a survey was developed for all Master of Engineering students (assessing courses at all three Arizona universities), the instrument was customized for the fiber optics course [3]. This effort was devoted specifically to capture the unique local perceptions. In that way, it was context sensitive. The formative assessment was designed to provide information for the purpose of improving the virtual learning environment.

The survey was developed to quantitatively measure student perceptions of the course experience with a series of statements using a five-point Likert scale: 1 = Strongly Disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, and 5 = Strongly Agree (See Appendix). Survey content validity was established by having the pilot questions reviewed by a team of experts that included engineering faculty, faculty who previously taught online courses, and instructional technology and assessment professionals. Additionally, the survey was piloted during the 1999 Spring and 1999 Fall semesters prior to the Fall 2000 fiber optics administration. Although that survey provided attitudinal measurement, openended questions yielding qualitative data shed further light on areas of assessment interest (such as undergraduate and graduate differences).

To assess course efficacy, we used conventional measures of success at the university level such as course grades and course completion. The latter was measured by whether or not the student took the final exam.

A. Participants

Survey population was identified as all students enrolled in the fiber optics course during the Fall 2000 semester. The population lacked homogeneity in terms of age, academic grade level, ASU student status, and location. Overall, there was a 100 percent response rate as a result of the survey administration process.

B. Data Analysis

Data were collected and imported into SPSS (Statistical Package for the Social Sciences) and Microsoft Excel for analysis. Students were coded to reveal differing variables (i.e., gender and ethnicity; and undergraduate and graduate status). Nonparametric tests [4] were conducted for special groups to examine differences between groups (i.e., undergraduate and graduate students) within the class due to small sample sizes and because the data were in ordinal form. T-tests were conducted for course grade point average comparisons. Finally, qualitative data were coded to examine for recurring themes, patterns, and discrepancies between groups.

IV. EMPIRICAL RESULTS OF STUDY

There were three primary empirical results of the study. First, despite the fact that the students enrolled in what they believed to be a traditional classroom, 40 percent migrated to the Web delivery. Significant differences existed between students who chose to view the course online and those who did not. More specifically, "Web watchers" (those who migrated to the Web for the recorded lectures and associated class materials) were more satisfied than "non-Web watchers" (those who used video recordings of the lectures to view materials) with technical aspects of the course, student communication, and schedule accommodation. Web watchers were more likely to take another course on the Web. Additionally, there were significant differences between graduate and undergraduate students. Finally, the course retention rate was 95 percent, which is considered to be exceptional for an online course.