Outcome measures for emergency medicine residency graduates: do measures of academic and clinical performance during residency training correlate with American Board of Emergency Medicine test performance?

Abstract

Objectives: Emergency medicine (EM) residency programs are increasingly asked to have measurable outcomes of residents' performance. Successful completion of the written and oral American Board of Emergency Medicine (ABEM) examinations is one key outcome. In the clinical practice of EM, emergency physicians (EPs) are often measured by their clinical productivity (patients per hour). This study explored the correlation between these measures of academic and clinical performance and hypothesized that clinical productivity would have a positive association with ABEM performance.

Methods: A prospective written survey was sent to all EPs completing training at an established Midwest 3-year EM residency program between 1994 and 2005 (53,000 annual visits in 1994 to 65,000 annual visits in 2005). Physicians self-reported their national ABEM written and oral board scores in a blinded fashion. Simulated oral board scores and senior written in-training examination scores were also recorded. Postgraduate Year 3 (PGY3) clinical productivity was calculated as annual patient encounters divided by hours worked. Correlations among these variables were assessed by Pearson's correlation coefficient, with p < 0.05 being considered statistically significant. Multiple regression analysis was performed for ABEM oral and written examination scores.

Results: Fifty-six of 85 residents responded to the initial survey. There was no significant correlation between clinical productivity and ABEM scores, either written (r = -0.021, p = 0.881) or oral (r = -0.02, p = 0.879). There was also no significant correlation between productivity and simulated oral board scores (r = 0.065, p = 0.639) of PGY3 in-training scores (r = 0.078, p = 0.57). As previously reported, there were positive and significant correlations between PGY3 in-service scores and ABEM written examination scores (r = 0.60, p < 0.0001), as well as ABEM oral and written examination scores (r = 0.51, p < 0.0001). Multiple regression analysis revealed only the PGY3 in-training examination was a significant predictor of the ABEM oral and written scores (p < 0.001).

Conclusions: PGY3 resident clinical productivity, when measured as patients per hour, correlated poorly with academic performance when measured by written and oral ABEM scores. The PGY3 in-training examination was predictive of the ABEM written and oral examination scores.