A nomogram to predict the probability of passing the American Board of Internal Medicine examination

Abstract

Background: Although the American Board of Internal Medicine (ABIM) certification is valued as a reflection of physicians' experience, education, and expertise, limited methods exist to predict performance in the examination.

Purpose: The objective of this study was to develop and validate a predictive tool based on variables common to all residency programs, regarding the probability of an internal medicine graduate passing the ABIM certification examination.

Methods: The development cohort was obtained from the files of the Cleveland Clinic internal medicine residents who began training between 2004 and 2008. A multivariable logistic regression model was built to predict the ABIM passing rate. The model was represented as a nomogram, which was internally validated with bootstrap resamples. The external validation was done retrospectively on a cohort of residents who graduated from two other independent internal medicine residency programs between 2007 and 2011.

Results: Of the 194 Cleveland Clinic graduates used for the nomogram development, 175 (90.2%) successfully passed the ABIM certification examination. The final nomogram included four predictors: In-Training Examination (ITE) scores in postgraduate year (PGY) 1, 2, and 3, and the number of months of overnight calls in the last 6 months of residency. The nomogram achieved a concordance index (CI) of 0.98 after correcting for over-fitting bias and allowed for the determination of an estimated probability of passing the ABIM exam. Of the 126 graduates from two other residency programs used for external validation, 116 (92.1%) passed the ABIM examination. The nomogram CI in the external validation cohort was 0.94, suggesting outstanding discrimination.

Conclusions: A simple user-friendly predictive tool, based on readily available data, was developed to predict the probability of passing the ABIM exam for internal medicine residents. This may guide program directors' decision-making related to program curriculum and advice given to individual residents regarding board preparation.

Fig. 1

Predictive graphic nomogram for the probability of passing the American Board of Internal Medicine (ABIM) exam.

Fig. 2

Screenshot of nomogram Web site.

Fig. 3

Calibration plot for predicted and observed passing ABIM examination. Residents in the validation cohort were divided in five quintiles based on the predicted probability. The 45-degree straight line represents ideal agreement between actual and predicted probability. The vertical bars represent the 95% CI of the actual probability. Note that the points for the three highest quintiles were overlapping together in the plot because of ignorable differences.