Predicting failure before it happens: A 5-year, 1042 participant prospective study

Abstract

Purpose of the article: Students who fail assessments are at risk of negative consequences, including emotional distress and cessation of studies. Identifying students at risk of failure before they experience difficulties may considerably improve their outcomes.

Methods: Using a prospective design, we collected simple measures of engagement (formative assessment scores, compliance with routine administrative tasks, and attendance) over the first 6 weeks of Year 1. These measures were combined to form an engagement score which was used to predict a summative examination sat 14 weeks after the start of medical school. The project was repeated for five cohorts, giving a total sample size of 1042.

Results: Simple linear regression showed engagement predicted performance (R²_adj = 0.03, F(1,1040) = 90.09, p < 0.001) with a small effect size. More than half of failing students had an engagement score in the lowest two deciles.

Conclusions: At-risk medical students can be identified with some accuracy immediately after starting medical school using routinely collected, easily analysed data, allowing for tailored interventions to support students. The toolkit provided here can reproduce the predictive model in any equivalent educational context. Medical educationalists must evaluate how the advantages of early detection are balanced against the potential invasiveness of using student data.

Keywords: Assessment; psychometrics; student support.