An Experimental Study of the Effectiveness of Collaborative Testing in an Entry-Level Computer Programming Class

Journal of Information Systems Education, Fall 2005 by Simkin, Mark G

ABSTRACT

In collaborative testing environments, students work together in small groups to answer examination questions. This study tested the hypothesis that group exams help student testing performance in IS classes. Quantitative and qualitative analyses of student scores on two examinations (a quiz and a formal, extensive midterm) found significantly higher group scores (compared to individual scores), and that superior group performance was particularly notable for the constructed-response portion of the midterm. Both direct observation of the group process and a survey of student perceptions about the groupexam process suggested that there were (1) few of the behavioral problems often attributed to group exams, (2) objective conflict resolution, and (3) favorable student perceptions of the process itself. This paper also provides several caveats that should be considered when interpreting these findings and suggests several avenues for future research.

Keywords: Collaborative testing, group testing, participative learning, computer programming, information systems education

1. INTRODUCTION

The term collaborative testing refers to a number of pedagogical tools that instructors can use to assess student understanding of course materials. Common to all of them is the requirement that students provide one common set of exam answers, presumably representing the collective wisdom of the group's members. Common formats for collaborative test questions include multiple-choice, true/false, or fill-in-the-blank:, although such "constructed response questions" as short-answer or even essay questions are possible. Constructed-response questions are probably the most difficult to administer on a group basis and therefore not commonly used.

Proponents of group exams argue that such assessments better reflect the reality of the work place-i.e., the fact that such professionals as auditors, IT personnel, and advertising analysts typically work in teams, must create a single set of deliverables, and are judged on the basis of group, rather than individual, performance. If "teamwork" is so common to such diverse occupations and work settings, why should academic practice be any different?

At the university level, experiments in group exams have been conducted in a variety of academic settings, including classes in accounting, marketing, mathematics, and tax (see, respectively, Cottell and Millis, 1993; Mclntyre, et. al., 1999; Berry and Nyman, 2002; and Kite, 1996). Less experimentation appears in the field of information systems-a surprising absence in light of the team orientation involved in so much of the industry's activities. Accordingly, the author sought to address this deficiency with some experimental trials in an entry-level computer programming class.

The next section of this paper discusses collaborative testing in greater detail and the rationale and concerns for the process. The third section of this paper describes a set of experiments and a survey that the author conducted to assess the effectiveness of group exams in an IS class setting. That section of the paper also provides the results of these experiments and describes the insights the author gained from this experiment. The fourth section of the paper provides some additional observations about the group-exam process and also describes some caveats that limit the findings described here or that might hamper the use of group exams in other classes. The final portion of this paper provides a summary of this work and conclusions about it.

2. COOPERATIVE LEARNING AND GROUP EXAMS

Much of the rationale for group exams rests in the moregeneral arguments for cooperative learning. Although teachers have used this term to encompass everything from group work on business cases to teamwork on intramural sports activities, Johnson, et. al. (1991) suggest more rigorous criteria, which include: (1) positive dependence, in which positive rewards occur only when the group as a whole, rather than the individuals within it, succeed, (2) face-to-face interaction, (3) social norms of behavior requiring individual accountability for group performance, (4) the need for collaborative skills such as oral communication skills (including the ability to present cogent logic in persuasive arguments), leadership qualities, and organizational skills, and (5) the ability to work with, and help, others in order to accomplish group goals. The term "group" is not explicitly defined among these criteria, thereby allowing "cooperative learning" to apply to student sets as small as two individuals or as large as an entire class. In practice, groups of from three to six students appear to be the most common.

2.1 The Rationale for Cooperative Learning Pedagogy

One key argument favoring cooperative learning is motivation (Graham and Graham, 1997; Hite, 1996; Astin, 1993). This argument rests on the idea that students who are forced to perform tasks in group settings are also positively motivated to share what they know, listen to the ideas of others, critically examine evidence, and actively search for objective methods or knowledge sources for choosing among several alternative approaches or solutions. King (1992) also notes that working in groups motivates the participants, to find impartial ways of resolving conflicts-an important social skill in almost any professional setting. Finally, Astin (1993) suggests that students may be motivated to study harder and therefore learn more-for example, because they will be answerable to their peers for wrong answers.