Factors affecting likelihood of applicants being offered a place in medical schools in the United Kingdom in 1996 and 1997: retrospective study

British Medical Journal, Oct 24, 1998 by I.C. McManus, Aneez Esmail, Marie Demetriou

Introduction

The importance of annually monitoring the admissions procedure within each medical school cannot be over-emphasised. It is naturally desirable that factors such as medical parents, social class, private education, race, and sex do not become significant predictors of shortlisting or interview success, whether unconsciously determined or otherwise.[1]

The Council of Heads of Medical Schools has recently decided to place anonymised data for individual applications to medical schools in the United Kingdom on the website of the Universities and Colleges Admissions Service (www.ucas.ac.uk). For the first time much of the information that is important in selecting medical students can be examined in detail. The data set is large and complex (4.5 million pieces of information on 93 000 applications from 19 000 candidates over two years). In this paper I summarise measures that are related to receiving an offer at individual medical schools and overall. These measures have been important in previous studies[2-5] or are of intrinsic interest. Not all potentially important measures were available. For example, computer readable data were not available on grades at GCSE (general certificate of secondary education, the examinations generally taken at the age of 16); predicted grades at A level (the examinations generally taken at the age of 18); the textual statements provided by applicants and referees; or the selection process within medical schools, including shortlisting and interviewing, which are often used to assess motivation and personality. A necessarily brief paper cannot do complete justice to such rich data, and a more detailed report is available on the website of the Universities and Colleges Admissions Service (www.ucas.ac.uk); computer programs for carrying out additional statistical analyses are available on request from me.

The two aims of this paper are to provide an overview of measures important in selection and to look in detail at the controversial issue of the selection of applicants from ethnic minority groups.[6-9] Important methodological concerns, to avoid the flaws of some previous studies,[10 11] are adequate control of confounding variables and analysis of otters rather than entrants (since candidates themselves select between schools when holding several offers[12]).

Methods

The Universities and Colleges Admissions Service provided anonymised data for all applications to medical school in the 1996 and 1997 cycles--that is, applications during autumn 1995 and 1996--for applicants with permanent residence in the United Kingdom (home applicants). During 1996 and 1997 applicants could make a maximum of six applications (although they were recommended to make five). Deferred entry (or the so called gap year) was known only for the 1997 data. The service sent non-anonymised data to medical schools for checking. Ethnic origin was self classified by applicants using standard census categories.

Data were supplied as an EXCEL 5.0 workbook, which was analysed using SPSS for Windows 6.1 (syntax and system files are available on request from me). Separate SPSS files were produced for analyses of candidates and applications.

Statistical analysis used multiple logistic regression with simultaneous adjustment of each variable for all other factors. The dependent variable for the analysis of candidates was receipt of one or more offers and for the analysis of applications receipt of an offer. Missing values were handled by imputation of population means.[13]

A levels and Scottish highers--Most applicants to medical schools outside Scotland take A levels after application, whereas about halt of applicants to Scottish medical schools take highers, which are taken before application. Selectors typically have no information on achieved grades at A level but only GCSE grades and predicted grades at A level, which were not available in this study. In this study, however, academic achievement is described in terms of achieved A level grades, the current standard of sixth form educational achievement. As in previous studies,[2 4 5] academic achievement is summarised as the number of A levels and highers taken and the mean grade attained in A levels and highers.

Significance testing--Assessing statistical significance is potentially problematic when about 21 factors are assessed at 27 individual schools in two separate years, giving some 1100 tests. A straightforward solution for avoiding et inflation (type I error) treats as significant only results nominally significant in both years, defined as a nominal significance level of [is less than] 0.01, or [is less than] 0.05 on one occasion prod a geometric mean less than 0.01--for example, P = 0.05 and P = 0.002. My unpublished report describes many additional analyses, which cannot be described fully in this short paper. When claims seem not to be fully supported by evidence further information is available in the report (available at www.ucas.ac.uk).

Results

Selection overall--Table 1 summarises for the 18 943 candidates in the combined 1996-7 data the relation between the 24 variables and receipt of one or more offers. Eighteen predictors were significant with P [is less than] 0.05 and 17 with P [is less than] 0.001.