Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Oct 3;23(1):720.
doi: 10.1186/s12909-023-04689-0.

Relationship between epa level of supervision with their associated subcompetency milestone levels in pediatric fellow assessment

Richard B Mink  1 , Carol L Carraccio  2 , Bruce E Herman  3 , Pnina Weiss  4 , David A Turner  2 , Diane E J Stafford  5 , Kathleen A McGann  6 , Jennifer Kesselheim  7 , Deborah C Hsu  8 , Pamela C High  9   10 , Jill J Fussell  11 , Megan L Curran  12 , Patricia R Chess  13 , Cary Sauer  14 , Sarah Pitts  15 , Angela L Myers  16 , John D Mahan  17 , Christiane E L Dammann  18 , Tandy Aye  5 , Alan Schwartz  19 ; Subspecialty Pediatrics Investigator Network
Collaborators, Affiliations

Relationship between epa level of supervision with their associated subcompetency milestone levels in pediatric fellow assessment

Richard B Mink et al. BMC Med Educ. .

Abstract

Background: Entrustable Professional Activities (EPA) and competencies represent components of a competency-based education framework. EPAs are assessed based on the level of supervision (LOS) necessary to perform the activity safely and effectively. The broad competencies, broken down into narrower subcompetencies, are assessed using milestones, observable behaviors of one's abilities along a developmental spectrum. Integration of the two methods, accomplished by mapping the most relevant subcompetencies to each EPA, may provide a cross check between the two forms of assessment and uncover those subcompetencies that have the greatest influence on the EPA assessment.

Objectives: We hypothesized that 1) there would be a strong correlation between EPA LOS ratings with the milestone levels for the subcompetencies mapped to the EPA; 2) some subcompetencies would be more critical in determining entrustment decisions than others, and 3) the correlation would be weaker if the analysis included only milestones reported to the Accreditation Council for Graduate Medical Education (ACGME).

Methods: In fall 2014 and spring 2015, the Subspecialty Pediatrics Investigator Network asked Clinical Competency Committees to assign milestone levels to each trainee enrolled in a pediatric fellowship for all subcompetencies mapped to 6 Common Pediatric Subspecialty EPAs as well as provide a rating for each EPA based upon a 5-point LOS scale.

Results: One-thousand forty fellows were assessed in fall and 1048 in spring, representing about 27% of all fellows. For each EPA and in both periods, the average milestone level was highly correlated with LOS (rho range 0.59-0.74; p < 0.001). Correlations were similar when using a weighted versus unweighted milestone score or using only the ACGME reported milestones (p > 0.05).

Conclusions: We found a strong relationship between milestone level and EPA LOS rating but no difference if the subcompetencies were weighted, or if only milestones reported to the ACGME were used. Our results suggest that representative behaviors needed to effectively perform the EPA, such as key subcompetencies and milestones, allow for future language adaptations while still supporting the current model of assessment. In addition, these data provide additional validity evidence for using these complementary tools in building a program of assessment.

Keywords: Assessment; Correlation; EPAs; Equations; Fellow; Milestones.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

Fig. 1
Fig. 1
Schematic showing the relationship of the Lead an Interprofessional Healthcare Team EPA with mapped core competencies and subcompetencies. Eight subcompetencies map to this EPA. Milestones were created for all subcompetencies but of the 8, only 5 are reported to the Accreditation Council for Graduate Medical Education. Personal & Professional Development (PPD) is a core competency unique to Pediatrics. Abbreviations: PC = patient care, PBLI = practice-based learning and improvement, ICS = interpersonal and communication skills, SBP = system-based practice
Fig. 2
Fig. 2
Graph showing Spearman Rho correlations (95% confidence intervals) of EPA level of supervision ratings by the Clinical Competency Committee with unweighted (#1) and weighted (#2) composite score using all mapped milestones, unweighted using only the ACGME reported milestones (#3) and unweighted using milestones from when the fellowship program director was (#4) or was not (#5) a member of the Clinical Competency Committee. The last graph in each group (#6) shows the correlation of EPA level of supervision ratings made independently by the fellowship program director with milestones for when the program director was not a member of the clinical competency committee. Data are from the fall 2014 and spring 2015. Abbreviations: ACGME = Accreditation Council for Graduate Medical Education, CCC = Clinical Competency Committee, EPA = Entrustable professional activities, FPD = fellowship program director; LOS = level of supervision
Fig. 3
Fig. 3
ROC curves and area under the curve for spring 2015 data using unweighted Accreditation Council for Graduate Medical Education subcompetency milestone composite score to achieve EPA level of supervision ratings of 4 or 5 (solid line; black) or only 5 (dashed line; red). Ratings utilized data from all members of the Clinical Competency Committee. Area under the curve in black is based upon a rating of 4 or 5 while that in red used a rating of 5. Abbreviations: ROC = receive operating characteristics, AUC = area under the curve, EPA = Entrustable professional activities
See this image and copyright information in PMC

References

    1. Batalden P, Leach D, Swing S, Dreyfus H, Dreyfus S. General competencies and accreditation in graduate medical education. Health Aff. 2002;21(5):103–111. doi: 10.1377/hlthaff.21.5.103. - DOI - PubMed
    1. Frank JR, Danoff D. The Can MEDS initiative: implementing an outcomes-based framework of physician competencies. Med Teach. 2007;29:642–647. doi: 10.1080/01421590701746983. - DOI - PubMed
    1. Edgar L, McLean S, Hogan S, Hamstra S, Holmboe E. The Milestones Guidebook. Version 2020. Available at https://www.acgme.org/globalassets/milestonesguidebook.pdf. Accessed 13 May 2023.
    1. Tekian A, Hodges BD, Roberts TE, Schuwirth L, Norcini J. Assessing competencies using milestones along the way. Med Teach. 2015;37(4):399–402. doi: 10.3109/0142159X.2014.993954. - DOI - PubMed
    1. Englander R, Frank JR, Carraccio C, Sherbino J, Ross S, Snell L, on behalf of ICBME Collaborators. Toward a shared language for competency-based education. Med Teach. 2017;39(6):582–587. - PubMed