Difficulties with anonymous shortlisting of medical school applications and its effects on candidates with non-European names: prospective cohort study

British Medical Journal, Jan 8, 2000 by Andrew B Lumb, Andy Vail

Comparison of blind and open scoring

The differences between blind and open total scores did not significantly differ from zero in any of the three groups (fig 2). Using linear regression to control for confounding factors as above, the unidentified ethnic minority group had similar differences between blind and open scores to the European group (0.00, -0.20 to 0.20). The identified ethnic minority group had larger differences of blind minus open scores than the European group (0.25, - 0.04 to 0.53), points compatible with minor positive discrimination, but this was not statistically significant (P = 0.09).

Analysis of component scores

Analysis of the components of the blind score showed the identified ethnic minority group to be less likely than the other groups to drop marks for the career insight component (table 2). In each of the other components the pattern was that the European candidates were least likely to drop marks, with the identified and unidentified ethnic minority groups performing similarly.

Table 2 Number (%) of candidates dropping marks on each component
of blinded score

                                 European
                                  group      Unidentified ethnic
Component                        (n=1483)   minority group (n=398)

Career insight(*)                579 (39)          180 (45)
Non-academic
 activities([dagger])            564 (38)          200 (50)
Academic
 profile([double dagger])        630 (42)          222 (56)
Career suitability([sections])   573 (39)          175 (44)

                                   Identified ethnic
Component                        minority group (n=166)

Career insight(*)                       42 (25)
Non-academic
 activities([dagger])                   85 (51)
Academic
 profile([double dagger])               92 (55)
Career suitability([sections])          75 (45)

Dropped: (*) >1 of 4 points; ([dagger]) >2 of 6 points; ([double dagger]) any of 4 points; ([sections]) >1 of 6 points.

The differences observed between component scores for the ethnic groups were assessed for differential effects between blind and open scoring--that is, for discrimination affected by anonymising the application form (table 3). For the career insight component blinding was found to have a differential effect across the groups. Whereas European applicants were approximately equally likely to drop marks on either assessment, fewer applicants from unidentified ethnic minority groups dropped marks on open assessment and fewer applicants from identified ethnic minority groups scored worse on blind assessment. There was no evidence of other such differential effects for either nonacademic activities, academic profile, or career suitability.

Table 3 Comparison between discordant blind and open component
scores. Values are numbers of applicants

                           European   Unidentified ethnic
Component                   group        minority group

Career insight:
 Good open, poor blind       263               99
 Poor open, good blind       240               51
Non-academic activities:
 Good open, poor blind       210               53
 Poor open, good blind       210               68
Academic profile:
 Good open, poor blind       158               40
 Poor open, good blind       157               40
Career suitability:
 Good open, poor blind       240               50
 Poor open, good blind       253               72

                           Identified ethnic
Component                    minority group

Career insight:
 Good open, poor blind             16
 Poor open, good blind             27
Non-academic activities:
 Good open, poor blind             24
 Poor open, good blind             26
Academic profile:
 Good open, poor blind             18
 Poor open, good blind             18
Career suitability:
 Good open, poor blind             27
 Poor open, good blind             28