Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Observational Study
. 2019 Mar;19(2):144-151.
doi: 10.1016/j.acap.2018.06.002. Epub 2018 Jun 18.

Milestone Ratings and Supervisory Role Categorizations Swim Together, but Is the Water Muddy?

Daniel J Schumacher  1 , Kathleen W Bartlett  2 , Sean P Elliott  3 , Catherine Michelson  4 , Tanvi Sharma  5 , Lynn C Garfunkel  6 , Beth King  7 , Alan Schwartz  8 ; APPD LEARN CCC Study Group
Affiliations
Observational Study

Milestone Ratings and Supervisory Role Categorizations Swim Together, but Is the Water Muddy?

Daniel J Schumacher et al. Acad Pediatr. 2019 Mar.

Abstract

Objective: This single-specialty, multi-institutional study aimed to determine 1) the association between milestone ratings for individual competencies and average milestone ratings (AMRs) and 2) the association between AMRs and recommended supervisory role categorizations made by individual clinical competency committee (CCC) members.

Methods: During the 2015-16 academic year, CCC members at 14 pediatric residencies reported milestone ratings for 21 competencies and recommended supervisory role categories (may not supervise, may supervise in some settings, may supervise in all settings) for residents they reviewed. An exploratory factor analysis of competencies was conducted. The associations among individual competencies, the AMR, and supervisory role categorizations were determined by computing bivariate correlations. The relationship between AMRs and recommended supervisory role categorizations was examined using an ordinal mixed logistic regression model.

Results: Of the 155 CCC members, 68 completed both milestone assignments and supervision categorizations for 451 residents. Factor analysis of individual competencies controlling for clustering of residents in raters and sites resulted in a single-factor solution (cumulative variance: 0.75). All individual competencies had large positive correlations with the AMR (correlation coefficient: 0.84-0.93), except for two professionalism competencies (Prof1: 0.63 and Prof4: 0.65). When combined across training year and time points, the AMR and supervisory role categorization had a moderately positive correlation (0.56).

Conclusions: This exploratory study identified a modest correlation between average milestone ratings and supervisory role categorization. Convergence of competencies on a single factor deserves further exploration, with possible rater effects warranting attention.

Keywords: clinical competency committee; entrustment; graduate medical education; milestone-based assessment; pediatrics.

PubMed Disclaimer

Publication types