Developing a comprehensive assessment program for engineering education

Journal of Engineering Education, Oct 1998 by McGourty, Jack, Sebastian, Catherine, Swart, William

Developing a Comprehensive Assessment Program for Engineering Education*

ABSTRACT

This paper provides an overview of one institution's efforts to establish a comprehensive assessment program for continuous improvement of engineering education. A five step systematic process to develop an integrated assessment program from identifying educational objectives to applying measurement methods is explained in detail. Activities to encourage faculty participation and commitment are outlined. Four integrated assessment processes used by both faculty and students to assess and provide performance feedback are described. The focus of these assessment methods is on the measurement, development, and improvement of student learning outcomes aligned with ABET Engineering Criteria 2000. Preliminary results and lessons learned from the overall experience are highlighted.

I. INTRODUCTION

During the last five years, New Jersey Institute of Technology (NJIT) has undergone a comprehensive transformation of its undergraduate engineering education system. This transformation was supported through two major grants: A National Science Foundation Grant to Support the Gateway Undergraduate Engineering Education Coalition, and a grant from the Defense Advanced Projects Research Agency under its Technology Re-Investment Program. The strategies pursued to accomplish this transformation included comprehensive curricular reform, the development of distance learning technologies as well as the use of the internet and computer-based training programs, the integration of industry in the undergraduate education process, and a comprehensive assessment program supporting continuous improvement.

The implementation and institutionalization of this transformation of undergraduate engineering education is underway at NJIT. An integral component of institutionalization and associated culture change is a comprehensive assessment program designed to support continuous improvement. This program was developed by a team comprised of faculty, administration, and an assessment specialist who were charged to design and implement a formal, valid, and useful assessment and continuous improvement process.

This paper will describe the steps that the institution has taken to imbed assessment and continuous improvement into the educational environment. The implementation of several strategies is discussed and results ofthese activities are highlighted.

II. OUTCOME-DRIVEN ASSESSMENT

The assessment of student learning outcomes has become a primary focus for higher education institutions in today's competitive environment. There is a great deal of pressure from industry, state legislatures, and academic accreditation entities to incorporate sound assessment techniques into all educational programs.l For example, the Accreditation Board for Engineering and Technology (ABET) has incorporated program outcomes and assessment as a key criterion in its Engineering Criteria 2000. According to the ABET criteria, the focus of an institution's assessment efforts should be on the measurement of student learning outcomes in a systematic and valid manner.2

In general, outcome-driven assessment processes provide critical information to faculty and administrators on the effectiveness of the design, delivery, and direction of an educational program. Students must also receive feedback on their development and performance throughout their education. Effective assessment processes must answer the following question: Have the program's graduates acquired the knowledge and skills defined by predetermined educational objectives and outcomes? Few educational institutions have a comprehensive system for measuring program results in terms of student learning outcomes. There are some exceptions; for example, Alverno College, located in Milwaukee, Wisconsin, has created a system that assesses the degree to which all students can demonstrate specific abilities throughout their educational experience.3

III. ASSESSMENT PROGRAM DESIGN

The NJIT focus on assessment is an integral part of the overall program to transform the current engineering curriculum to meet the needs of today's and future practicing engineers. Assessment processes, to be effective, must be aligned with the institution's mission and goals for the future. For NJIT, this means that the assessment program must operate within a learning environment that is focused on developing professionals and technological leaders for a global economy. Assessment activities must take into account not only the acquisition of knowledge, but its application as well. In addition, assessment processes must be designed for faculty working in a multi-disciplinary and highly integrative learning environment. Finally, results from assessment processes need to be applied for continuous improvement of student learning outcomes and program effectiveness.

An assessment team, comprised of engineering, humanities, and computer science faculty, administration, and an assessment expert was chartered to "institute a formal, valid, and useful assessment and continuous improvement process." During the early stages of the curriculum transformation, assessment was founded largely on traditional methodologies (i.e., student ratings of instructor effectiveness). The team's short-term strategies were threefold: 1) establish a structured approach to identify educational objectives and student learning outcomes associated with EC-2000, 2) create a set of assessment processes to measure defined objectives and student learning outcomes, and 3) establish faculty development activities designed to create enthusiasm and "buy-in." These strategies were implemented during an eighteen month period.4