Difficulties with anonymous shortlisting of medical school applications and its effects on candidates with non-European names: prospective cohort study

British Medical Journal, Jan 8, 2000 by Andrew B Lumb, Andy Vail

Abstract

Objective To assess the feasibility of anonymous shortlisting of applications for medical school and its effect on those with non-European names.

Design Prospective cohort study.

Setting Leeds school of medicine, United Kingdom. Subjects 2047 applications for 1998 entry from the United Kingdom and the European Union.

Intervention Deletion of all references to name and nationality from the application form.

Main outcome measures Scoring by two admissions tutors at shortlisting.

Results Deleting names was cumbersome as some were repeated up to 15 times. Anonymising application forms was ineffective as one admissions tutor was able to identify nearly 50% of candidates classed as being from an ethnic minority group. Although scores were lower for applicants with non-European names, anonymity did not improve scores. Applicants with non-European names who were identified as such by tutors were significantly less likely to drop marks in one particular non-academic area (the career insight component) than their European counterparts.

Conclusions There was no evidence of benefit to candidates with non-European names of attempting to blind assessment. Anonymising application forms cannot be recommended.

Introduction

In the United Kingdom there is huge competition to study medicine, with in excess of 13 000 applications through the Universities and Colleges Admissions Service for just over 4000 places. A series of recent studies has found that the likelihood of success is less among applicants from ethnic minority groups than among white applicants.[1-6] A study based on 1991 entry indicated that the situation was improving.[5] Even so, when seven other mainly academic aspects of the application were taken into account not being from an ethnic minority group remained a significant predictor of success. A recent study looking at all home applicants for entry in 1996 and 1997 found a greater disadvantage for applicants from ethnic minority groups than previously.[6] This study was, however, limited for technical reasons by not being able to include data on GCSE grades, which form a large part of the selection process and which were important predictors of success in previous studies.[5]

Most unsuccessful applicants are rejected solely on assessment of their application form--that is, at the `shortlisting' stage before being invited for interview. It is during shortlisting that students from ethnic minority groups are believed to be disadvantaged.[5] The application form contains no explicit reference to the applicant's ethnic background, so it seems likely that any discrimination must be based on the applicant's name. For this reason it has been suggested that the whole of the shortlisting process be performed anonymously.[5]

We decided to assess the feasibility of assessing forms anonymously within the current admissions system of the Universities and Colleges Admissions Service. In addition, we assessed the impact of doing so on the shortlisting system we have used at Leeds school of medicine for the past four years.

Methods

Shortlisting process

Our shortlisting process involves each application form being assessed separately by two of three admissions tutors (including AL). A score from zero to 20 points is awarded made up of four components including career insight (4 points), non-academic activities (6 points), academic profile (4 points), and suitability for a medical career as described by the confidential reference (6 points). When assessing applications, admissions tutors are unaware of the other selector's score. The sum of the two scores then forms the sole basis of the decision to reject, accept, or interview the applicant, although the threshold values may vary throughout the year according to the numbers and quality of applications received.

Anonymising forms

We studied all home (including EU) applicants to the medical course for entry in 1998 except for graduate applications, which are assessed separately. An admissions clerk, who played no part in the assessment of applicants, anonymised forms. All text to be deleted was first overwritten with a red marker pen, which allowed the text underneath to be easily read, and this comprised the `open' application. The form was then photocopied, which rendered the highlighted section indecipherable, and this comprised the `blind' form. The following text was deleted: full name, email address, country of birth, applicant's signature, and all references to the applicant's name found in the personal statement or confidential reference sections. Batches of about 100 forms with alternate blind and open applications were sent to each selector, with care being taken to avoid any selector assessing the same applications twice. In this way selectors always read alternate open and blind forms, of different applicants, and each form was assessed both open and blind by two different selectors. The short time available for processing of applications and other commitments by selectors prohibits equity in the number of forms assessed and equal randomisation of pairs of selectors.