Correlation of Performance on the ENTRUST Assessment Platform With Other Variables in Competency-Based Surgical Education

Abstract

Objective: With the implementation of American Board of Surgery (ABS) Entrustable Professional Activities (EPAs), there is continued need for objective, evidence-based assessment tools to augment existing microassessments and inform readiness for entrustment. The ENTRUST Assessment Platform is an online virtual-patient simulation platform to assess trainees' surgical decision-making competence across preoperative, intraoperative, and postoperative phases of care. This study collects additional validity evidence for the ENTRUST platform in its relationship to other established variables in competency-based surgical education.

Design: This is a prospective analysis of surgical resident performance on the ENTRUST Right Lower Quadrant (RLQ) pain/Appendicitis EPA Assessment. ENTRUST scores were analyzed by PGY-level and correlations with Accreditation Council for Graduate Medical Education (ACGME) Case Logs, ACGME Surgery Milestones, and ABS In-Service Training Examination (ABSITE) scores were evaluated. Bivariate analyses were performed using Spearman rank correlations.

Setting: This study was conducted at a tertiary academic center (Stanford University, Palo Alto, CA) in a proctored exam setting.

Participants: Thirty-two PGY-1 though PGY-5 general surgery residents completed the ENTRUST RLQ Pain/Appendicitis EPA Assessment containing four case scenarios which were iteratively developed and scored by expert consensus and aligned with ABS EPA definitions.

Results: ENTRUST grand total score was positively correlated with PGY-level (rho = 0.57, p = 0.001), ACGME appendectomy case log volume (rho = 0.55, p = 0.002), and ABSITE raw score (rho = 0.66, p = 0.0004). ENTRUST performance was significantly correlated with all eighteen ACGME Surgery Milestones (rho = 0.43 to rho = 0.54, all p≤0.01), with the strongest correlation seen for PC1 (Patient Evaluation and Decision Making) (rho = 0.54, p = 0.006).

Conclusions: Performance on ENTRUST was significantly correlated with established variables in surgical training, including ACGME Appendectomy Case Logs, ABSITE, and ACGME Surgery Milestones. This study strengthens existing validity evidence for the ENTRUST Assessment Platform as an objective assessment of clinical decision-making. ENTRUST is an assessment tool which can augment microassessments and support competency-based medical education.

Keywords: Assessment; clinical decision-making; competency-based medical education; entrustable professional activity; graduate medical education; patient simulation.