A Randomized Controlled Trial of a Deep Language Learning Model-Based Simulation Tool for Undergraduate Medical Students in Surgery

Abstract

Introduction: Effective communication is a critical skill for surgeons that commences often with history-taking. While simulation-based training is utilized to enhance these skills, recent advancements in artificial intelligence (AI), especially deep language learning models (DLM), offer new opportunities. This study evaluates the integration of DLM as a simulated patient (SP) into surgical history-taking training for senior medical students during clinical rotations.

Methods: A randomized controlled trial was conducted with surgery module students. Participants were divided into control and intervention groups, the former receiving standard experiential learning and the latter adding 3 structured sessions with DLM (ChatGPT, Open AI) as SP (with interaction texts submitted for tutor evaluation). All students underwent Objective Structured Clinical Examination (OSCE) of history-taking with a human SP and blinded assessor blinded by group for baseline competency ascertainment and again after either intervention or a similar time of standard learning. Intervention group students were anonymously surveyed to assess communication confidence and perspectives on DLM as SP.

Results: After initial pilot trialing, ninety students participated formally with 45 assigned to each arm via randomized cluster sampling. DLM-content was uniformly appropriate. Baseline scores were similar but significantly increased in the intervention group alone (p < 0.001, 0.37v0.19 Cohen D education effect size). 62% of students completed the survey, a majority (57%) articulating increased confidence, rich detail in DLM histories (72%) and would use again (95%).

Conclusions: DLM effectively enhanced surgical history-taking skills. These findings indicate AI can serve as a valuable tool for student development alongside clinical learning.

Keywords: ChatGPT; artificial Intelligence (AI); deep learning language model; history taking; medical education; simulation.