Exploring UK medical school differences: the MedDifs study of selection, teaching, student and F1 perceptions, postgraduate outcomes and fitness to practise

Abstract

Background: Medical schools differ, particularly in their teaching, but it is unclear whether such differences matter, although influential claims are often made. The Medical School Differences (MedDifs) study brings together a wide range of measures of UK medical schools, including postgraduate performance, fitness to practise issues, specialty choice, preparedness, satisfaction, teaching styles, entry criteria and institutional factors.

Method: Aggregated data were collected for 50 measures across 29 UK medical schools. Data include institutional history (e.g. rate of production of hospital and GP specialists in the past), curricular influences (e.g. PBL schools, spend per student, staff-student ratio), selection measures (e.g. entry grades), teaching and assessment (e.g. traditional vs PBL, specialty teaching, self-regulated learning), student satisfaction, Foundation selection scores, Foundation satisfaction, postgraduate examination performance and fitness to practise (postgraduate progression, GMC sanctions). Six specialties (General Practice, Psychiatry, Anaesthetics, Obstetrics and Gynaecology, Internal Medicine, Surgery) were examined in more detail.

Results: Medical school differences are stable across time (median alpha = 0.835). The 50 measures were highly correlated, 395 (32.2%) of 1225 correlations being significant with p < 0.05, and 201 (16.4%) reached a Tukey-adjusted criterion of p < 0.0025. Problem-based learning (PBL) schools differ on many measures, including lower performance on postgraduate assessments. While these are in part explained by lower entry grades, a surprising finding is that schools such as PBL schools which reported greater student satisfaction with feedback also showed lower performance at postgraduate examinations. More medical school teaching of psychiatry, surgery and anaesthetics did not result in more specialist trainees. Schools that taught more general practice did have more graduates entering GP training, but those graduates performed less well in MRCGP examinations, the negative correlation resulting from numbers of GP trainees and exam outcomes being affected both by non-traditional teaching and by greater historical production of GPs. Postgraduate exam outcomes were also higher in schools with more self-regulated learning, but lower in larger medical schools. A path model for 29 measures found a complex causal nexus, most measures causing or being caused by other measures. Postgraduate exam performance was influenced by earlier attainment, at entry to Foundation and entry to medical school (the so-called academic backbone), and by self-regulated learning. Foundation measures of satisfaction, including preparedness, had no subsequent influence on outcomes. Fitness to practise issues were more frequent in schools producing more male graduates and more GPs.

Conclusions: Medical schools differ in large numbers of ways that are causally interconnected. Differences between schools in postgraduate examination performance, training problems and GMC sanctions have important implications for the quality of patient care and patient safety.

Keywords: Fitness to practise; GMC sanctions; Institutional histories; Medical school differences; National Student Survey; National Training Study; Postgraduate qualifications; Preparedness; Problem-based learning; Teaching styles.

Fig. 1

Descriptive statistics for non-PBL schools, PBL schools and all schools (mean, median, SD and N) for the measures used in the analyses. Means are compared using t tests, allowing for different variances, with significant values indicated in bold (p < 0.05). Significant differences are also shown in colour, red indicating the group with the numerically higher score and green the lower scores. Note that higher scores do not always mean better

Fig. 2

Correlogram showing correlations of the 50 measures across the 29 medical schools. Positive correlations are shown in blue and negative in red (see colour scale on the right-hand side), with the size of squares proportional to the absolute size of the correlation. Significance levels are shown in two ways: two asterisks indicate correlations significant with a Tukey-adjusted correction, and one asterisk indicates correlations significant with p < 0.05. For abbreviated variable names, see text. Measures are classified in an approximate causal order with clusters separated by blue lines

Fig. 3

Correlogram of the 24 measures associated with particular specialties across the 29 medical schools. Correlations are the same as in Fig. 1, but re-ordered so that the different specialties can be seen more clearly. Specialties are separated by the horizontal and vertical blue lines, with examination and non-examination measures separated by solid green lines. Two asterisks indicate within- and between-specialty correlations that meet the appropriate Tukey-adjusted p value; one asterisk indicates correlations that meet a conventional 0.05 correlation without correction

Fig. 4

Teaching measures in relation to outcome measures. Regression lines in dark blue are for all points, pale blue for all points excluding imputed values and green for all points excluding Oxbridge (blue circles). Yellow boxes around points indicate PBL schools. See text for discussion

Fig. 5

Structural model of the causal relationships of the 29 measures in Supplementary Fig. 2. Very thick lines indicate Bayes factor (BF) > 100, thick lines BF > 30, medium lines BF > 10 and thin lines BF > 3, with black and red indicating positive and negative relationships respectively. Beta and BF are shown alongside each path. For further details see text

Fig. 6

Reduced structural model for performance in postgraduate examinations indicating direct and indirect strong effects on postgraduate performance. All other paths have been removed. For further details, see text

Fig. 7

Scatterplots of relations between outcome measures and entry grades. For details of plots, see Fig. 4

Fig. 8

Scatterplots of relations between NSS measures and postgraduate exam performance. For details of plots, see Fig. 4