Assessing the Math Online tool: A progress report

Mathematics and Computer Education, Fall 2003 by McSweeney, Laura, Weiss, Joan

Introduction

The goal of the Math Online system is to enable students to become more active learners of mathematics and to reinforce basic algebraic and calculus skills. Via randomly generated multiple-choice questions delivered through a web-based system, students practice various computational techniques and take proctored online quizzes outside of class time. The assessment of the Math Online system was done using data collected from a randomized comparative study conducted in introductory calculus courses during two academic years, but could be adapted to other courses. Data analysis is presented as well as the lessons we learned about using technology outside of the classroom.

1. What is Math Online?

Math Online is a Foundation supported web based interactive system created by Mathematics faculty at Fairfield University for the delivery of multiple-choice questions to reinforce and provide practice of algebraic and computational skills. In addition, a student's knowledge of these skills can be assessed via proctored online quizzes.

2. Objectives of Math Online

Most instructors will concur that many students in their calculus classes flounder due to the students' weaknesses in basic algebraic and precalculus skills. Excessive class time can be spent reviewing algebraic rules and formulas, practicing computational skills, and assessing students' knowledge of these rules and skills.

The main objective of the Math Online system is to move much of this algebraic review and computational skill practice online and out of the classroom. Students become active learners and take control of their learning. Via Math Online they can practice skills and get immediate feedback outside the classroom at any time on any computer with web access and they can practice a skill until they master it. Proctored online quizzes verify students' mastery of these skills.

Moving most of the learning and assessment of basic skills outside the classroom realizes a second goal of freeing in-class time for alternate activities.

3. Description of Math Online

In 1999 the Davis Educational Foundation provided $200,000 to the Fairfield University Mathematics and Economics Departments for a three-year project to restructure introductory mathematics and economics courses. The Mathematics Department also received additional funding from the E.L. Cord Foundation. Mathematics Professors Christopher Bernhardt, Laura McSweeney, Stephen Sawin, and Joan Weiss developed the Math Online web based system for the mathematics component of the project. Mathematica, a computer algebra system, was used to generate the random multiple-choice questions. Professor Sawin developed the bulk of the question generating system in Perl. The other project participants coded questions and constructed help sheets. Professor McSweeney coordinated the assessment of the effectiveness of Math Online. Professor Weiss incorporated Math Online into a section of her Applied Calculus I and II course during the 2000 - 01 academic year.

Students in the Math Online sections are required to complete a predetermined number of proctored online quizzes and receive credit for them when they achieve a score greater than a preset minimum mastery score. To prepare for these proctored online graded quizzes, during 2000-01, the first year of implementation, students would practice skills by completing sample quizzes. If they were unsuccessful in passing a practice quiz they could seek help from tutors available at scheduled times or from their instructor. In the second year students practiced skills via online assignments that consisted of sample quiz questions. If a student was unable to master the skill tested in an assignment they could request help via online Help Pages that pointed out the specific type of error they had made and referred them to the appropriate section(s) in their text book. Further assistance was again available via tutors and their instructor.

On a daily basis Math Online randomly generates numerous versions of questions for each skill. When a quiz is requested Math Online randomly selects questions for the quiz based on the skills to be tested so that each quiz is essentially unique. The proctored quizzes usually consist of six to ten multiple choice questions testing about four to six skills. Students are required to complete the proctored quizzes outside of class in the Math Center. Proctors, typically sophomore through senior mathematics majors, make sure each student does not use a text, notes or receive assistance with a quiz, and after verifying the student's ID, enter a code to allow the student to take a quiz in proctored mode. Proctors also tutor students who have questions while they are practicing skills.

The Math Online system also has course management capabilities. These features include instructor-constructed quizzes and assignments, detailed information about individual student performance on each assignment and quiz, and an online grade book with secure student access.