What Predicts Performance? A Multicenter Study Examining the Association Between Resident Performance, Rank List Position, and United States Medical Licensing Examination Step 1 Scores

Abstract

Background: Each application cycle, emergency medicine (EM) residency programs attempt to predict which applicants will be most successful in residency and rank them accordingly on their program's Rank Order List (ROL).

Objective: Determine if ROL position, participation in a medical student rotation at their respective program, or United States Medical Licensing Examination (USMLE) Step 1 rank within a class is predictive of residency performance.

Methods: All full-time EM faculty at Los Angeles County + University of Southern California (LAC + USC), Harbor-UCLA (Harbor), Alameda Health System-Highland (Highland), and the University of California-Irvine (UCI) ranked each resident in the classes of 2013 and 2014 at time of graduation. From these anonymous surveys, a graduation ROL was created, and using Spearman's rho, was compared with the program's adjusted ROL, USMLE Step 1 rank, and whether the resident participated in a medical student rotation.

Results: A total of 93 residents were evaluated. Graduation ROL position did not correlate with adjusted ROL position (Rho = 0.14, p = 0.19) or USMLE Step 1 rank (Rho = 0.15, p = 0.14). Interestingly, among the subgroup of residents who rotated as medical students, adjusted ROL position demonstrated significant correlation with final ranking on graduation ROL (Rho = 0.31, p = 0.03).

Conclusions: USMLE Step 1 score rank and adjusted ROL position did not predict resident performance at time of graduation. However, adjusted ROL position was predictive of future residency success in the subgroup of residents who had completed a sub-internship at their respective programs. These findings should guide the future selection of EM residents.

Keywords: Rank Order List (ROL); USMLE Step 1 scores; medical student rotation; predicting resident performance; recruitment.