Introducing an assessment tool based on a full set of end-of-training EPAs to capture the workplace performance of final-year medical students

Abstract

Background: While literature on the theoretical value of entrustable professional activities (EPAs) for assessment is rapidly expanding, little experience exists on its application. The aims of this study are to develop and explore the utility of an EPA-based assessment tool for capturing the workplace performance of final-year medical students based on a full set of end-of-training EPAs.

Methods: The tool was developed in a systematic iterative process. Twelve 12 end-of-undergraduate medical training EPAs were nested into 72 smaller EPAs and cross-mapped onto a 6-point supervision level scale, both adjusted to the context of final-year clerkships. One version was created for students' self-assessment of their ability to carry out tasks and their history of carrying out tasks, and another version was created for supervisors' assessment of students' ability to carry out tasks. The tool was administered to final-year clerkship students and their clinical supervisors to explore its utility as an assessment approach. The results were analysed using descriptive and interferential statistics.

Results: We enrolled a total of 60 final-year medical students. For 33 students, ratings were provided from one supervisor and for 27 students from two supervisors. With regard to the reliability and validity of the tool, students' and supervisors' ratings showed an overall good internal consistency as well as variability between and within the EPAs. Over the full EPA range, students rated their ability to perform a task slightly higher than their task performance history and slightly lower than the supervisors' ratings. Students' self-ratings of their ability to perform a task correlated with their history in performing the task. Supervisors' ratings correlated among supervisors and not with students' ratings. Concerning educational outcomes, supervisors' average rating of students' ability to perform the EPAs without direct supervision was 64%, and key findings being double-checked.

Conclusions: This study introduces a tool that is adjusted to the final-year clerkship context and can assess the workplace performance of trainees based on a full set of end-of-training EPAs. Its utility characteristics suggest that the tool may be employed as a formative and outcome-aligned approach to the assessment of final-year students before entering into residency.

Keywords: Assessment; EPAs; Final-year clerkship; Workplace performance.

Fig. 1

Clerkship students’ self-ratings on the ability to perform a task (I can) and the history in performing a task (I did) as well as the supervisors’ rating on the students´ ability to perform the task (the student can) in relation to a 6-point supervision level scale (1–6). Shown are the results of 13 task of EPA 2. Perform general procedures of a physician. The bars depict the mean plus standard deviation. Significant differences were found in one of the 13 medical procedures (give an infusion, Mixed model, F(2, 65.038) = 5.190, p = 0.008, I can versus I did: p < 0.003, d = − 0.075, I can versus the student can: p = 0.003, d = − 0,494)