Assessing Distance Education Courses and Discipline Differences in their Effectiveness - 1 - Statistical Data Included

Journal of Instructional Psychology, March, 2001 by Paula Szulc Dominguez, Dennis R. Ridley

This research illustrated a new, parsimonious model that investigators interested in distance education can use to ask meaningful questions about the relative quality of distance education courses (Dominguez & Ridley, 1999). The approach removed the emphasis from student-level data and placed it upon course-based data. Sample data comparing online and traditional higher education courses covering nine disciplines were reported. These data revealed that preparation for advanced courses was statististically equivalent whether the course prerequisites were online courses or their traditional classroom counterparts. The article further explored the usefulness of this framework for identifying a significant discipline-related difference in the relative effectiveness of online and traditional prerequisites as preparation for advanced courses.

In this article, we have further explored an alternative framework for assessing distance education courses (Dominguez & Ridley, 1999). The article reviews the rationale for our new approach and presents a new analysis with updated data to demonstrate an application beyond our earlier presentation. The new application explores an apparent departmental difference found in the new analysis.

Rationale

Student performance in the here and now of a distance education course is typically at the center of assessment of their effectiveness. That is, investigators usually attend to how well students score on tests, exams, and assignments within the context of the distance education course itself. If comparisons with student performance in traditional classroom settings are to be made, they generally involve courses taken contemporaneously with the distance education course. In this way, assessments of distance education programs have amassed results that resemble a series of snapshots looking at the program and student performance on a semester-by-semester basis.

Using the student-level data within a particular time frame, institutions, distance education programs, and individual faculty have created a detailed portrait of distance education students and have established the comparability of student learning between distance education and traditional settings. This is a good beginning. Now that institutions have overcome the initial hurdles of establishing the first-generation distance education programs, the need arises for more elaborate, action-oriented information.

Focusing on student-level data tells only a limited tale. For example, generating a profile of the "successful" distance education student does not provide institutions with practical information for program improvement or refinement. What can an institution do with this piece of data? It is anathema in higher education to deny students entry into a course based on their demographic profile. Indeed, pushing the envelope of students' abilities is at the core of instruction. Neither does the information really help out individual faculty members interested in improving distance education students' performance. Faculty members simply do not have the power to age a student five years, produce several offspring for them, or boost their G.P.A. half a point. What else can be done to provide institutions and faculty members with action-oriented information about the quality of instruction found in distance education programs compared with instruction in on-campus classes?

We propose a two-pronged shift in distance education investigations. The first shift removes the emphasis on distance education students and places it on the course itself. The second expands the scope of investigations to include distance education students' subsequent performance in other classes. Using these parameters, such an assessment would question how well distance education courses prepare students for further study. Moreover, such an approach would allow institutions to compare student preparation in distance education settings versus their preparation in traditional education settings. This is very useful information for institutions that are expanding their distance education offerings. This assessment model has been used successfully to consider transfer between community colleges and four-year institutions (Quanty, Dixon, & Ridley, 1998).

Method

The data for this study were obtained from Christopher Newport University (CNU), a state-supported institution in Virginia that has offered online courses since the early 1990s. It is important to note that, unlike the situation at some other institutions, there really is not a distinct "online student" group at CNU. Instead, online students at CNU are subsumed in the larger, traditional student population. Most of the online students typically take a combination of online and traditional courses.

Every online course at CNU has an on-campus counterpart, which offers the opportunity for comparison. Many online courses are 100- and 200-level courses that act as prerequisites for more advanced courses, some of which are also offered online and others through traditional means. For the purpose of this study, "traditional" forms of prerequisites included on-campus courses, courses transferred in from another college or a community college, or credit given for performance on an examination. Using this information, we can describe four pathways to enrollment in advanced courses that are available to CNU students.