Application of AI Communication Training Tools in Medical Undergraduate Education: Mixed Methods Feasibility Study Within a Primary Care Context

Abstract

Background: Effective communication is fundamental to high-quality health care delivery, influencing patient satisfaction, adherence to treatment plans, and clinical outcomes. However, communication skills training for medical undergraduates often faces challenges in scalability, resource allocation, and personalization. Traditional methods, such as role-playing with standardized patients, are resource intensive and may not provide consistent feedback tailored to individual learners' needs. Artificial intelligence (AI) offers realistic patient interactions for education.

Objective: This study aims to investigate the application of AI communication training tools in medical undergraduate education within a primary care context. The study evaluates the effectiveness, usability, and impact of AI virtual patients (VPs) on medical students' experience in communication skills practice.

Methods: The study used a mixed methods sequential explanatory design, comprising a quantitative survey followed by qualitative focus group discussions. Eighteen participants, including 15 medical students and 3 practicing doctors, engaged with an AI VP simulating a primary care consultation for prostate cancer risk assessment. The AI VP was designed using a large language model and natural voice synthesis to create realistic patient interactions. The survey assessed 5 domains: fidelity, immersion, intrinsic motivation, debriefing, and system usability. Focus groups were used to explore participants' experiences, challenges, and perceived educational value of the AI tool.

Results: Significant positive responses emerged against a neutral baseline, with the following median scores: intrinsic motivation 16.5 of 20.0 (IQR 15.0-18.0; d=2.09, P<.001), system usability 12.0 of 15.0 (IQR 11.5-12.5; d=2.18, P<.001), and psychological safety 5.0 of 5.0 (IQR 5.0-5.0; d=4.78, P<.001). Fidelity (median score 6.0/10.0, IQR 5.2-7.0; d=-0.08, P=.02) and immersion (median score 8.5/15.0, IQR 7.0-9.8; d=0.25 P=.08) were moderately rated. The overall Immersive Technology Evaluation Measure scores showed a high positive learning experience: median 47.5 of 65.0 (IQR 43.0-51.2; d=2.00, P<.001). Qualitative analysis identified 3 major themes across 11 subthemes, with participants highlighting both technical limitations and educational value. Participants valued the safe practice environment and the ability to receive immediate feedback.

Conclusions: AI VP technology shows promising potential for communication skills training despite the current realism limitations. While it does not yet match human standardized patient authenticity, the technology has achieved sufficient fidelity to support meaningful educational interactions, and this study identified clear areas for improvement. The integration of AI into medical curricula represents a promising avenue for innovation in medical education, with the potential to improve the quality and effectiveness of training programs.

Keywords: artificial intelligence; communication skills; simulation; technology-enhanced learning; virtual patient.

Figure 1.. Participant recruitment and study protocol. AI: artificial intelligence; PSA: prostate-specific antigen.

Figure 2.. Violin plot of raw domain scores versus a neutral benchmark.

Figure 3.. Thematic map displaying the interlinking themes and subthemes. AI: artificial intelligence.