Assessing cognitive skills of pharmacy students in a biomedical sciences module using a classification of multiple-choice item categories according to bloom's taxonomy

American Journal of Pharmaceutical Education, Winter 2001 by Knecht, Kathryn T

The purpose of this study was to assess the use of multiple-choice examinations in testing cognitive skills of pharmacy students. Multiple-choice exam questions for an integrated Biomedical Sciences module were classified according to question structure and level of cognitive ability, and student performance was compared between and within question classifications. Student performance was significantly higher for questions of factual recall than for questions testing explanation or application skills. Within a given question classification, question topic and authorship were overall not significant determinants of student performance. Greater disparity in performance among questions of different classification was noted for lower-achieving students than for higher-achieving students. Knowledge of specific deficiencies in student performance may be helpful in shaping future course instruction and assessment.

INTRODUCTION

Multiple-choice exams have in general been criticized as inadequate for the assessment of upper-level cognitive skills. However, a number of authors have demonstrated that multiple-choice exams can be useful in assessing critical thinking(1-- 5). The testing of large numbers of students in a class (typically > 150 for this institution) is certainly simplified by the use of multiple-choice exams and electronic grading machines. It is worth asking, however, if the multiple-choice exams in current use in our institution adequately measure the cognitive skills that pharmacists will be expected to practice, especially as increasing expectations of clinical problem-solving are imposed.

In 1956, Bloom and colleagues(6) published their classic hierarchical taxonomy of knowledge, which meticulously classifies cognitive skills from the acquisition of knowledge through levels of comprehension, application, analysis, synthesis, and evaluation. Jacobs and Chase(5) and Gronlund(3) further characterized these cognitive levels and offered examples of exam questions that can be used to assess these levels of thinking. Ebel(4) described a simpler classification of seven kinds of tasks (terminology, factual information, generalization, explanation, calculation, prediction, recommended action). Whatever hierarchy may be used, pharmacy education and indeed education in general should encourage higher levels of cognitive skills, and student assessment should include questions that address each of these levels of understanding. This report addresses several questions regarding student exam performance during an integrated Biomedical Sciences module:

1. Did student performance on exam questions depend upon the level of understanding addressed? e.g., For a given content area, did students perform differently on knowledge vs. explanation questions?

2. Did patterns of student performance depend on level of student achievement? e.g., Did the above conclusions hold for both high-and low-achieving students?

3. Was student performance on a final exam representative of student performance on other exams? e.g., Did students perform differently on a final exam vs. a regular biweekly exam?

4. What effects did question content and authorship have on student performance in a given question category? e.g., Could students explain the autonomic nervous system as taught by one instructor as well as the renal system taught by a different instructor?

METHODS

The Biomedical Sciences module is a two-quarter integrated course that introduces basic mechanisms of pathophysiology, pharmacology, toxicology, and medicinal chemistry. Students take this eight-credit (per quarter) module concurrent with an eight-credit Pharmaceutical Sciences module and a two-credit Patient Care module at the beginning of their fourth year in a 0-6 pharmacy program. General education and basic science courses, including biochemistry, microbiology, and immunology, are prerequisites. After successful completion of introductory modules, students begin the first of six Therapeutic Modules (Cardiovascular, Infectious Disease, CNS, Endocrine, Oncology and GI, and Pharmacy Administration). Examination of student performance in the second quarter of the fourth year of the program allowed for assessment prior to the Therapeutic modules but after time for adjustment to the modular system.

Acknowledgements. The author would like to acknowledge the colleagues who have shared test questions, references, advice and support; especially Dr. Karen Kier, Dr. Margaret Zank, Dr. Margaret Cullen, Dr. Bobby Bryant, and Dr. Barbara Brandt. Thanks also to Mrs. Kathy Williams and Mrs. Kathy Sehlhorst for expert secretarial assistance, and to Melody Bivens for help with editing.

References

(1) Fuhrman, M., "Developing good multiple-choice tests and test questions," J. Geosci. Educ., 44, 379-384 (1996).

(2) Haladyna, T.M., Writing Test Items to Evaluate Higher Order Thinking, Allyn & Bacon Publishing, Needham Heights MA (1997).

(3) Gronlund, N.E,, Constructing Achievement Tests, 2nd edition, PrenticeHall, Inc., Englewood Cliffs NJ (1977)

(4) Ebel, R.L., Essentials of Educational Measurement, 3rd edition, Prentice-Hall, Inc., Englewood Cliffs NJ (1979)

(5) Jacobs, L.C. and Chase, C.I., Developing and Using Tests Effectively: A Guide for Faculty. Jossey-Bass Inc., San Francisco CA (1992).

(6) Bloom, B.S. editor, Taxonomy of Educational Objectives,: Cognitive Domain, David McKay Co., Inc,, New York NY (1956)