Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2016 Feb;8(1):50-6.
doi: 10.4300/JGME-D-15-00065.1.

Predicting Performance on the American Board of Physical Medicine and Rehabilitation Written Examination Using Resident Self-Assessment Examination Scores

Alex Moroz, Heejung Bang

Predicting Performance on the American Board of Physical Medicine and Rehabilitation Written Examination Using Resident Self-Assessment Examination Scores

Alex Moroz et al. J Grad Med Educ. 2016 Feb.

Abstract

Background: Studies across medical specialties have shown that scores on residency self-assessment examinations (SAEs) can predict performance on certifying board examinations.

Objective: This study explored the predictive abilities of different composite SAE scores in physical medicine and rehabilitation and determined an optimal cut-point to identify an "at-risk" performance group.

Methods: For our study, both predictive scores (SAE scores) and outcomes (board examination scores) are expressed in national percentile scores. We analyzed data in graduates of a physical medicine and rehabilitation residency program between 2008 and 2014. We compared mean, median, lowest, highest, and most recent score among up to 3 SAE scores with respect to their associations with the outcome via linear and logistic regression. We computed regression/correlation coefficient, P value, R (2), area under the curve, sensitivity, specificity, and predictive values. Identification of optimal cut-point was guided by accuracy, discrimination, and model-fit statistics.

Results: Predictor and outcome data were available for 88 of 99 residents. In regression models, all SAE predictors showed significant associations (P ≤ .001) and the mean score performed best (r = 0.55). A 1-point increase in mean SAE was associated with a 1.88 score increase in board score and a 16% decrease in odds of failure. The rule of mean SAE score below 47 yielded the highest accuracy, highest discrimination, and best model fit.

Conclusions: Mean SAE score may be used to predict performance on the American Board of Physical Medicine and Rehabilitation-written examination. The optimal statistical cut-point to identify the at-risk group for failure appears to be around the 47th SAE national percentile.

PubMed Disclaimer

Conflict of interest statement

Conflict of interest: The authors declare they have no competing interests.

Figures

FIGURE 1a and b
FIGURE 1a and b
Histograms of Self-Assessment Examination (SAE) and Board Examination Scores
FIGURE 2
FIGURE 2
Scatter Plot of Self-Assessment Examination (SAE) and Board Examination Scores Abbreviations: DF, degrees of freedom; MSE, mean squared error.
See this image and copyright information in PMC

References

    1. Leigh TM, Johnson TP, Pisacano NJ. Predictive validity of the American Board of Family Practice In-Training Examination. Acad Med. 1990;65(7):454–457. - PubMed
    1. Replogle WH, Johnson WD. Assessing the predictive value of the American Board of Family Practice in-training examination. Fam Med. 2004;36(3):185–188. - PubMed
    1. Waxman H, Braunstein G, Dantzker D, Goldberg S, Lefrak S, Lichstein E, et al. Performance on the internal medicine second-year residency in-training examination predicts the outcome of the ABIM certifying examination. J Gen Intern Med. 1994;9(12):692–694. - PubMed
    1. Webb LC, Juul D, Reynolds CF, 3rd, Ruiz B, Ruiz P, Scheiber SC, et al. How well does the psychiatry residency in-training examination predict performance on the American Board of Psychiatry and Neurology Part I Examination? Am J Psychiatry. 1996;153(6):831–832. - PubMed
    1. Juul D, Schneidman BS, Sexson SB, Fernandez F, Beresin EV, Ebert MH, et al. Relationship between Resident-In-Training Examination in psychiatry and subsequent certification examination performances. Acad Psychiatry. 2009;33(5):404–406. - PubMed

Publication types