Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Jan 27;25(1):129.
doi: 10.1186/s12909-025-06719-5.

A systematic review of the impact of artificial intelligence on educational outcomes in health professions education

Eva Feigerlova  1   2   3   4 , Hind Hani  5   6 , Ellie Hothersall-Davies  7
Affiliations

A systematic review of the impact of artificial intelligence on educational outcomes in health professions education

Eva Feigerlova et al. BMC Med Educ. .

Abstract

Background: Artificial intelligence (AI) has a variety of potential applications in health professions education and assessment; however, measurable educational impacts of AI-based educational strategies on learning outcomes have not been systematically evaluated.

Methods: A systematic literature search was conducted using electronic databases (CINAHL Plus, EMBASE, Proquest, Pubmed, Cochrane Library, and Web of Science) to identify studies published until October 1st 2024, analyzing the impact of AI-based tools/interventions in health profession assessment and/or training on educational outcomes. The present analysis follows the PRISMA 2020 statement for systematic reviews and the structured approach to reporting in health care education for evidence synthesis.

Results: The final analysis included twelve studies. All were single centers with sample sizes ranging from 4 to 180 participants. Three studies were randomized controlled trials, and seven had a quasi-experimental design. Two studies were observational. The studies had a heterogenous design. Confounding variables were not controlled. None of the studies provided learning objectives or descriptions of the competencies to be achieved. Three studies applied learning theories in the development of AI-powered educational strategies. One study reported the analysis of the authenticity of the learning environment. No study provided information on the impact of feedback activities on learning outcomes. All studies corresponded to Kirkpatrick's second level evaluating technical skills or quantifiable knowledge. No study evaluated more complex tasks, such as the behavior of learners in the workplace. There was insufficient information on training datasets and copyright issues.

Conclusions: The results of the analysis show that the current evidence regarding measurable educational outcomes of AI-powered interventions in health professions education is poor. Further studies with a rigorous methodological approach are needed. The present work also highlights that there is no straightforward guide for evaluating the quality of research in AI-based education and suggests a series of criteria that should be considered.

Trial registration: Methods and inclusion criteria were defined in advance, specified in a protocol and registered in the OSF registries ( https://osf.io/v5cgp/ ).

Clinical trial number: not applicable.

Keywords: AI-based training and assessment; Artificial intelligence; Educational outcomes; Health professions; Learning theories and principles.

PubMed Disclaimer

Conflict of interest statement

Declarations. Ethics approval and consent to participate: Not applicable. Consent for publication: Not applicable. Competing interests: The authors declare no competing interests.

Figures

Fig. 1
Fig. 1
Flowchart
Fig. 2
Fig. 2
Key challenges of AI-powered educational strategies in health professions education highlighted by studies and suggestions for the future
See this image and copyright information in PMC

References

    1. Karabacak M, Ozkara BB, Margetis K, Wintermark M, Bisdas S. The Advent of Generative Language models in Medical Education. JMIR Med Educ. 2023;9:e48163. - PMC - PubMed
    1. Preiksaitis C, Rose C. Opportunities, challenges, and Future Directions of Generative Artificial Intelligence in Medical Education: scoping review. JMIR Med Educ. 2023;9:e48785. - PMC - PubMed
    1. Safranek CW, Sidamon-Eristoff AE, Gilson A, Chartash D. The role of large Language models in Medical Education: applications and implications. JMIR Med Educ 14 août. 2023;9:e50945. - PMC - PubMed
    1. Abbott KL, George BC, Sandhu G, Harbaugh CM, Gauger PG, Erkin O, et al. Natural Language Processing to Estimate Clinical Competency Committee Ratings. 2021;78(6):2046–51. - PubMed
    1. Johnsson V, Søndergaard MB, Kulasegaram K, Sundberg K, Tiblad E, Herling L, et al. Validity evidence supporting clinical skills assessment by artificial intelligence compared with trained clinician raters. Med Educ. 2024;58(1):105–17. - PubMed

Publication types

LinkOut - more resources