Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 May 31;13(1):2360137.
doi: 10.1080/28338073.2024.2360137. eCollection 2024.

Assessment of Entrustable Professional Activities Among Dutch Endocrine Supervisors

Joanne M de Laat  1   2 , Anouk N A van der Horst-Schrivers  3 , Natasha M Appelman-Dijkstra  4 , Peter H Bisschop  5 , Koen M A Dreijerink  6 , Madeleine L Drent  6 , Melanie M van de Klauw  7 , Wendela L de Ranitz  8 , Aline M E Stades  8 , Nike M M L Stikkelbroeck  1 , Henri J L M Timmers  1 , Olle Ten Cate  2
Affiliations

Assessment of Entrustable Professional Activities Among Dutch Endocrine Supervisors

Joanne M de Laat et al. J CME. .

Abstract

Entrustable Professional Activities (EPAs) are an important tool to support individualisation of medical training in a competency-based setting and are increasingly implemented in the clinical speciality training for endocrinologist. This study aims to assess interrater agreement and factors that potentially impact EPA scores. Five known factors that affect entrustment decisions in health profesions training (capability, integrity, reliability, humility, agency) were used in this study. A case-vignette study using standardised written cases. Case vignettes (n = 6) on the topics thyroid disease, pituitary disease, adrenal disease, calcium and bone disorders, diabetes mellitus, and gonadal disorders were written by two endocrinologists and a medical education expert and assessed by endocrinologists experienced in the supervision of residents in training. Primary outcome is the inter-rater agreement of entrustment decisions for endocrine EPAs among raters. Secondary outcomes included the dichotomous interrater agreement (entrusted vs. non-entrusted), and an exploration of factors that impact decision-making. The study protocol was registered and approved by the Ethical Review Board of the Netherlands Association for Medical Education (NVMO-ERB # 2020.2.5). Nine endocrinologists from six different academic regions participated. Overall, the Fleiss Kappa measure of agreement for the EPA level was 0.11 (95% CI: 0.03-0.22) and for the entrustment decision 0.24 (95% CI 0.11-0.37). Of the five features that impacted the entrustment decision, capability was ranked as the most important by a majority of raters (56%-67%) in every case. There is a considerable discrepancy between the EPA levels assigned by different raters. These findings emphasise the need to base entrustment decisions on multiple observations, made by a team of supervisors and enriched with factors other than direct medical competence.

Keywords: Entrustable professional activities; capability assessment; clinical speciality training; decision-making factors; endocrinology; entrustment decisions; interrater agreement.

PubMed Disclaimer

Conflict of interest statement

No potential conflict of interest was reported by the author(s).

Figures

Figure 1.
Figure 1.
Relationship between the average rater confidence level about a trainee and the mean difference in recommended supervision levels between raters.
See this image and copyright information in PMC

References

    1. Chen HC, ten Cate O.. Assessment through entrustable professional activities. In: Delany C, editor. Learning & teaching in clinical contexts: a practical guide. Chatswood: Elsevier Australia; 2018. p. 286–8.
    1. Hauer KE, Ten Cate O, Boscardin C, et al. Understanding trust as an essential element of trainee supervision and learning in the workplace. Adv Health Sci Educ Theory Pract. 2014;19(3):435–456. doi: 10.1007/s10459-013-9474-4 - DOI - PubMed
    1. ten Cate O, Scheele F. Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med. 2007;82(6):542–547. doi: 10.1097/ACM.0b013e31805559c7 - DOI - PubMed
    1. Ten Cate O, Hart D, Ankel F, et al. Entrustment decision making in clinical training. Acad Med. 2016;91(2):191–198. doi: 10.1097/ACM.0000000000001044 - DOI - PubMed
    1. Ten Cate O, Carraccio C, Damodaran A, et al. Entrustment decision making: extending Miller’s pyramid. Acad Med. 2021;96(2):199–204. doi: 10.1097/ACM.0000000000003800 - DOI - PubMed