Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Nov 10;31(1):67-73.
doi: 10.1007/s40670-020-01097-7. eCollection 2021 Feb.

A Novel USMLE Step 1 Projection Model Using a Single Comprehensive Basic Science Self-Assessment Taken During a Brief Intense Study Period

Stephen D Bigach  1   2 , Robert D Winkelman  2 , Jonathan C Savakus  2   3 , Klara K Papp  2
Affiliations

A Novel USMLE Step 1 Projection Model Using a Single Comprehensive Basic Science Self-Assessment Taken During a Brief Intense Study Period

Stephen D Bigach et al. Med Sci Educ. .

Abstract

Background: Comprehensive Basic Science Self-Assessments (CBSSAs) offered by the National Board of Medical Examiners (NBME) are used by students to gauge preparedness for the United States Medical Licensing (USMLE) Step 1. Because residency programs value Step 1 scores, students expend many resources attempting to score highly on this exam. We sought to generate a predicted Step 1 score from a single CBSSA taken several days out from a planned exam date to inform student testing and study plans.

Methods: 2016 and 2017 Step 1 test takers at one US medical school were surveyed. The average daily score improvement from CBSSA to Step 1 during the 2016 study period was calculated and used to generate a predicted Step 1 score as well as mean absolute prediction errors (MAPEs). The predictive model was validated on 2017 data.

Results: In total, 43 of 61 respondents totaling 141 CBSSAs in 2016 and 37 of 43 respondents totaling 122 CBSSAs in 2017 were included. The final prediction model was [Predicted Step 1 = 292 - (292 - CBSSA score) * 0.987527 ^ (number of days out)]. In 2016, the average difference between predicted and actual scores was -0.81 (10.2) and the MAPE was 7.8. In 2017, 88 (72.1%) and 118 (96.7%) of true Step 1 scores fell within one and two standard deviations of a student's predicted score. There was a MAPE of 7.7. Practice form used (p = 0.19, 0.07) and how far out from actual Step 1 it was taken (p = 0.82, 0.38) were not significant in either year of study.

Conclusion: This projection model is reasonable for students to use to gauge their readiness for Step 1 while it remains a scored exam and provides a framework for future predictive model generation as the landscape of standardized testing changes in medical education.

Keywords: Projection modeling; Step 1; Testing; USMLE.

PubMed Disclaimer

Conflict of interest statement

Conflict of InterestThe authors declare that they have no conflict of interest.

Figures

Fig. 1
Fig. 1
Predicted scores as a function of days out from Step 1 exam that a CBSSA was taken. In parentheses are the points increased per day
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Morrison CA, Ross LP, Fogle T, Butler A, Miller J, Dillon GF. Relationship between performance on the NBME comprehensive basic sciences self-assessment and USMLE step 1 for U.S. and Canadian medical school students. Acad Med. 2010;85(10 SUPPL). 10.1097/ACM.0b013e3181ed3f5c. - PubMed
    1. Sawhill A, Butler A, Ripkey D, et al. Using the NBME self-assessments to project performance on USMLE step 1 and step 2: impact of test administration conditions. Acad Med. 2004;79:S55–S57. doi: 10.1097/00001888-200410001-00017. - DOI - PubMed
    1. Prober CG, Kolars JC, First LR, Melnick DE. A plea to reassess the role of United States medical licensing examination step 1 scores in residency selection. Acad Med. 2016;91(1):12–15. doi: 10.1097/ACM.0000000000000855. - DOI - PubMed
    1. McGaghie WC, Cohen ER, Wayne DB. Are United States medical licensing exam step 1 and 2 scores valid measures for postgraduate medical residency selection decisions? Acad Med. 2011;86(1):48–52. doi: 10.1097/ACM.0b013e3181ffacdb. - DOI - PubMed
    1. Makdisi G, Takeuchi T, Rodriguez J, Rucinski J, Wise L. How we select our residentsa survey of selection criteria in general surgery residents. J Surg Educ. 2011;68:67–72. doi: 10.1016/j.jsurg.2010.10.003. - DOI - PubMed

LinkOut - more resources