Difficulties with anonymous shortlisting of medical school applications and its effects on candidates with non-European names: prospective cohort study

British Medical Journal, Jan 8, 2000 by Andrew B Lumb, Andy Vail

Assignment of applicants to ethnic group

Ethnic background was determined by two administrative clerks not concerned with the selection process. From the full name, applicants with non-European sounding names were classified as from an ethnic minority group and only coded as such when both clerks regarded the name as non-European. Although not a strict definition of ethnic background, this is the definition most relevant to the potential discrimination under investigation. For each applicant the following data were recorded: order of application, non-European name, sex, scores for individual components of the assessment by both selectors, and final outcome of the application (offer or reject). When assessing forms blind, selectors were asked to indicate whether they had identified the applicant as being from an ethnic minority group from information provided on the form.

Analysis of data

Data were analysed in three stages with SPSS. Firstly, to confirm that applicants from ethnic groups receive lower marks, total score was modelled by linear regression. Secondly, differences between blind and open scores for each individual were compared to assess the effect of blinding. Finally, component scores were dichotomised at about the overall median so that numbers of candidates dropping more marks than average could be analysed for each component. Our interest was only in discordant scores, and we had no prior view as to whether blinded scores would be better or worse than open scores. Poisson regression of discordant scores was used to assess whether blinded scores were more worse (or less better) in the applicants from ethnic minority groups, which would indicate the putative discrimination.

Results

In total 2047 applications were included in the analysis, of which 1485 (72.5%) were adjudged to have a European name by the administrative staff. Overall, 166 (290/o) forms (including two adjudged European) were identified as being from an applicant from an ethnic minority group by the selector assessing the anonymous form, whereas the remaining 398 were classified as unidentified ethnic minority group (table 1).

Table 1 Identification of blinding by scorer, and percentage of
candidates from ethnic minority groups identified by each blinded
scorer

                          Unidentified ethnic   Identified ethnic
Scorer   European group     minority group        minority group

A              512                103               94 (48)
B              403                 96               63 (40)
C              568                199                9 (4)
Total         1483                398              166 (29)(*)

(*) Includes two applicants with European names.

Association of ethnic group with total score

The mean (SD) of total score was 28.7 (4.6) points (fig 1). The corresponding figures for each group were: European 29.2 (4.3), unidentified ethnic minority 27.2 (5.1), and ethnic minority identified 27.7 (4.3). This group difference was highly statistically significant (P [is less than] 0.0001) by linear regression with or without adjustment for potentially confounding factors (sex, time of application, combination of scorers). Early application and being female (difference 1.3, 95% confidence interval 0.9 to 1.7 points) were also both significantly associated with higher scores.