Assessment of curricular competency outcomes

American Journal of Pharmaceutical Education, Fall 2001 by Kirkpatrick, Mary Ann F, Pugh, Carol B

This paper describes assessment initiatives to: (i) identify outcomes addressed in all courses of a new entry-level Doctor of Pharmacy program; and (ii) ascertain students' perceptions of progress toward educational outcomes. During the four years after the implementation of a new curriculum, course coordinators identified competencies and supporting competencies addressed in their courses as well as the extent of coverage. Students assessed their command of competencies and supporting competencies annually during the same time frame. Results included: (0) identification of content duplication and omissions; and (it) no significant differences in student self-assessments by class for each curricular year. With the addition of these surveys, we have established an assessment program that includes faculty members' judgments of their contribution to curricular content and students' appraisals of learning.

INTRODUCTION

The Virginia Commonwealth University (VCU) School of Pharmacy began a four-year entry level Doctor of Pharmacy Program in the fall of 1995. The curriculum for this program was predominantly based on desired outcomes developed by the American Association of Colleges of Pharmacy (ACPE) Center for the Advancement of Pharmaceutical Education (CAPE) Advisory Panel on Educational Outcomes in 1994(1). When the new professional degree program began, the Assessment Committee reviewed the School's existing assessment activities and concluded that the on-going assessment program needed to be augmented. First, we needed to ensure that all desired curricular outcomes were being addressed and that no outcomes were unnecessarily duplicated. Second, we needed to examine the new curriculum from a student perspective.

The Committee developed a two-part assessment strategy that would support the following goals: (i) to monitor curriculum content by identifying educational outcomes addressed in all courses; and (ii) to ascertain students' perceptions of personal progress toward mastery of the School's educational outcome objectives. These goals were addressed over a four-year period using two different survey tools. The outcomes assessed by this process included attitudes and skills, as well as knowledge. Although a large amount of data were generated, the curricular assessment was designed to meet programmatic needs, not to test hypotheses. The purpose of this paper is to present a summary of the results and lessons learned during the course of the curricular assessment.

METHODS

Two instruments were used to collect assessment data. The faculty instrument was used after the conclusion of each new year of the didactic curriculum. The entire student body completed the student survey on an annual basis.

Faculty Assessment of Course Coverage of Competencies The faculty instrument consisted of a list of 12 desired competencies or outcomes and 288 supporting or enabling outcomes approved by the faculty for the new four-year professional degree program. This list of 300 items was derived from the 1994 report of the CAPE Advisory Panel on Educational Outcomes(I), edited by the curriculum redesign task force, and approved by the faculty. Appendix A contains a sample of two of the competencies and their supporting outcomes (a copy of the complete set of competencies and supporting outcomes is available upon request from the corresponding author). Space was provided to the left of each outcome and enabling outcome in which course coordinators were asked to record the extent of coverage or extent to which each outcome was addressed in his or her courses. For example, a faculty member may have indicated that the material was included in a four-hour lecture sequence, or was covered in multiple case study exercises during the semester. Other possible types of responses included the nature of testing or evaluation used to determine if the student had mastered the competency. Because of the large amount of overlap among the supporting outcomes for competencies 8, 9, and 10, these three competency statements were combined into a single grouping during the design of the new curriculum. Thus, while there are 12 competency statements, only 10 sets of supporting outcomes are reported.

All faculty members who coordinated courses for pharmacy students, including those from the School of Medicine, were asked to participate in the course assessment process. Because of the burdensome nature of the instrument, surveys were completed only for the first offering of each required course in the didactic portion of the curriculum. A single copy of the survey was placed in the mailbox of each course coordinator; replacement copies were furnished upon request. Frequent reminders to complete the survey were made at department and school faculty meetings; stragglers received individual encouragement. All first-year course coordinators were asked to identify outcomes and the extent of outcome coverage in their courses at the end of the spring semester 1996. At the end of the spring semester in 1997, all secondyear course coordinators were asked to indicate the same information for their courses. Likewise, at the end of the spring semester in 1998, all third-year course coordinators were asked to identify the outcomes and extent of coverage in their courses. Clerkship preceptors were not asked to supply information on the outcomes and extent of coverage for their rotations. Over 250 practitioners participate in our clerkship program, providing a wide variety of learning experiences. Because of the diversity of clerkship sites and the variability of patient experiences within individual sites, we opted to focus on only the didactic portion of the curriculum.