ChatGPT and Other Large Language Models in Medical Education - Scoping Literature Review

Abstract

This review aims to provide a summary of all scientific publications on the use of large language models (LLMs) in medical education over the first year of their availability. A scoping literature review was conducted in accordance with the PRISMA recommendations for scoping reviews. Five scientific literature databases were searched using predefined search terms. The search yielded 1509 initial results, of which 145 studies were ultimately included. Most studies assessed LLMs' capabilities in passing medical exams. Some studies discussed advantages, disadvantages, and potential use cases of LLMs. Very few studies conducted empirical research. Many published studies lack methodological rigor. We therefore propose a research agenda to improve the quality of studies on LLM.

Keywords: Artificial intelligence; ChatGPT; Generative AI; Large language model; Medical education.

Fig. 1

Flow diagram of the literature review process. The figure, which was adapted from Page et al. [22], was slightly modified, since we relied exclusively on databases and registers for study identification

Fig. 2

Map of the corresponding authors’ origin. We found no publications from authors of countries highlighted in gray. A deeper shade of blue indicates more publications on LLMs in medical education; a lighter shade of blue indicates less (but not zero) publications