Validity of the in-training examination for predicting American Board of Internal Medicine certifying examination scores

Abstract

Objective: To determine whether the results of the Internal Medicine In-Training Examination (ITE) can predict subsequent performance on the American Board of Internal Medicine certifying examination (ABIMCE).

Design: Retrospective data review.

Setting: A mixture of six community hospital and university-based internal medicine training programs in the Eastern United States.

Subjects: 109 residents who first took the ABIMCE in 1988 or 1989, and who had also taken at least one ITE.

Measurements: Scores for the composite and subspecialty sections of the ITE were compared with those for the ABIMCE. An R2 was obtained to relate the scores on the two examinations. A cutoff score was derived to maximize the ability of the ITE to discriminate between residents who were likely to pass and those who were likely to fail the ABIMCE.

Main results: ABIMCE scores were available for 109 residents who had also taken the ITE during PGY-2 (19), PGY-3 (50), or both years (40). Composite scores on the ABIMCE were highly correlated with those on the ITE-PGY-2 (R2 = 0.593) and the ITE-PGY-3 (R2 = 0.677) (p less than 0.0001 for each). Most of the subspecialty sections on the two examinations were significantly correlated, although less strongly (range of R2 = 0.041 to 0.32) than were the composite scores. An empirically derived cutoff score of the 35th percentile on the ITE-PGY-2 had a positive predictive value of 89% (probability of passing ABIMCE) and a negative predictive value of 83% (probability of failing ABIMCE).

Conclusions: Performance on the ITE can accurately predict and is highly correlated with performance on the ABIMCE. ITE results may therefore be useful in counseling residents about their educational needs in preparation for the ABIMCE.