Mid-Year Medical Knowledge Milestones and ABSITE Scores in First-Year Surgery Residents

Abstract

Objective: Accreditation Council for Graduate Medical Education (ACGME) Surgery milestone ratings in the "Knowledge of Diseases and Conditions" (MK1) sub competency have been shown to correlate with American Board of Surgery In Training Examination (ABSITE) scores, and hypothesized to predict them. To better assess the predictive value of the MK1 milestone and avoid the potential bias caused by previous years' ABSITE scores, we designed a study including only first-year (PGY-1) residents and analyzed the correlation between their mid-year MK1 ratings and their scores in the ABSITE they took approximately a month later.

Methods: De-identified United States Medical Licensing Examination (USMLE) Step 1 and Step 2 scores, mid-year MK1 milestone ratings and the subsequent ABSITE standard scores for the five academic years from 2014-2015 to 2018-2019 were collected and tabulated for 247 PGY-1 preliminary- and categorical-track residents from ten ACGME-accredited surgery residency programs.

Results: The mid-year rating of PGY-1 residents' MK1 was predictive of their subsequent first ABSITE score for the entire cohort and for the categorical residents' subset. Notably, controlling for all other independent predictors, each half-point increase in MK1 rating was associated with a 25-point increase in ABSITE score. Preliminary residents performed significantly worse on the ABSITE, and their scores did not correlate significantly with their MK1 ratings.

Conclusions: The mid-year rating of PGY-1 residents' MK1 was predictive of their subsequent first ABSITE score for the entire cohort and for the categorical but not the preliminary residents. This finding suggests that evaluators correctly rated MK1 higher in the categorical residents who did perform better on the subsequent ABSITE.

Keywords: Medical Knowledge; Milestones; evaluations; medical knowledge; standardized testing; surgery residency.