Traditional student, nontraditional student, and pharmacy practitioner attitudes toward the use of standardized patients in the assessment of clinical skills

American Journal of Pharmaceutical Education, Spring 2000 by Monaghan, Michael S, Turner, Paul D, Vanderbush, Ross E, Grady, Angela R

Performance-based assessments in pharmaceutical education are needed. We developed a pharmacy objective structured clinical examination (P-OSCE), using standardized patients, that requires participants to demonstrate the ability to perform clinically-based professional practice competencies. The purpose of this report was to evaluate the assessment process from the perspective of the person being tested. Thirty-six traditional students, 13 nontraditional students, and 15 BS-degree pharmacy practitioners were evaluated using the P-OSCE. At the end of the examination, participants voluntarily completed a five-point Likert scale questionnaire measuring their attitudes toward the assessment process. Descriptive analysis was performed on returned questionnaires with response mode reported. Chi-Square analyses were performed on similar items as follow-up to significant findings. Statistically significant differences among groups occurred on only one item. Traditional students were less comfortable with their responses in the counseling/consulting phase of the testing than were their nontraditional and practitioner counterparts (x2 = 9.04, P = 0.011 ). The overall opinions of the participants were positive. All groups believed the P-OSCE reflected "real world" practice and indicated their areas of strengths and weaknesses. Practitioners believed most positively in the process while traditional students stated that this assessment should become part of the curriculum. All groups recommended its continued use. This information may be useful to other programs considering performance-based assessments.

INTRODUCTION

The term assessment generically applies to a wide range of approaches used to measure educational effectiveness(1). The American Council on Pharmaceutical Education's (ACPE) recently revised Accreditation Standards and Guidelines specifically address assessment in terms of student learning(2). The standard recommends that programs establish methods for both the formative and summative evaluation of student achievement. These assessments should measure cognitive learning, mastery of essential practice skills, and the ability to communicate effectively while using data in both critical thinking and problem solving processes. One type of assessment which meets these criteria is a performance assessment. Performance assessments require students to demonstrate or produce evidence of learning; in other words, the student must actively generate an original response rather than passively selecting one from a given list, as in multiple choice testing(3). An example of a performance-based assessment is the "Objective Structured Clinical Examination" (OSCE) approach utilizing standardized patients(4). Standardized patients are people trained to present a case in a standardized, consistent manner, without variation between encounters. Standardized patients have been employed for almost 30 years in academic medicine and such impressive data exist that standardized patients and the OSCE format will become part of the licensure process for medicine in North America.

During an OSCE, students rotate around a circuit of stations, each station centering around a case requiring the student to perform one or more clinical tasks(5). Cases are developed from real life experiences, similar to case studies or written simulations, but have the added benefit of human interaction within a short time frame. This OSCE approach has been adapted for pharmaceutical education(6). We developed a Pharmaceutical Care Encounters Program (PCEP) which formulates and administers pharmacy OSCEs (P-OSCEs). The POSCE requires students to demonstrate the ability to perform professional pharmacy practice competencies. The P-OSCE utilizes standardized patients in a summative evaluation of clinical skills associated with adult medicine experiential clerkships. This report describes a project to evaluate the performance-based examination from the perspective of the person being tested, namely the student.

BACKGROUND

A more detailed description of the PCEP, including background, validity and reliability data related to P-OSCEs exists(6). A brief summary will be provided here. The PCEP, the program which developed the P-OSCEs, was created to meet the need of objectively evaluating clinical pharmacy education not directly observed. Because of the demands of the entry-level PharmD program and nontraditional track, direct observation of students, whether traditional or nontraditional, was not always possible. Therefore, preceptors required a means of objectively and reliably assessing students' performance of professional practice competencies. The research team, composed of pharmacy practice faculty who were also preceptors, looked to other health care professions and identified the use of standardized patients and OSCEs as a well-published and reliable means of evaluating clinical skills. The reader is referred to two recent reviews which provide a historical overview of standardized patient use in academic medicine(7,8). When standardized patients are used in assessing clinical skills of medical students and medical residents, evaluation of the process by the trainee and faculty are positive(9,10).