Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 May;6(5):e367-e373.
doi: 10.1016/S2589-7500(24)00047-5.

Randomised controlled trials evaluating artificial intelligence in clinical practice: a scoping review

Ryan Han  1 , Julián N Acosta  2 , Zahra Shakeri  3 , John P A Ioannidis  4 , Eric J Topol  5 , Pranav Rajpurkar  6
Affiliations

Randomised controlled trials evaluating artificial intelligence in clinical practice: a scoping review

Ryan Han et al. Lancet Digit Health. 2024 May.

Abstract

This scoping review of randomised controlled trials on artificial intelligence (AI) in clinical practice reveals an expanding interest in AI across clinical specialties and locations. The USA and China are leading in the number of trials, with a focus on deep learning systems for medical imaging, particularly in gastroenterology and radiology. A majority of trials (70 [81%] of 86) report positive primary endpoints, primarily related to diagnostic yield or performance; however, the predominance of single-centre trials, little demographic reporting, and varying reports of operational efficiency raise concerns about the generalisability and practicality of these results. Despite the promising outcomes, considering the likelihood of publication bias and the need for more comprehensive research including multicentre trials, diverse outcome measures, and improved reporting standards is crucial. Future AI trials should prioritise patient-relevant outcomes to fully understand AI's true effects and limitations in health care.

PubMed Disclaimer

Conflict of interest statement

Declaration of interests EJT receives funding from the National Center for Advancing Translational Sciences/National Institutes of Health (grant number UL1TR002550). JNA is an employee of Rad AI, outside of the submitted work. RH receives funding from the National Institute of General Medical Sciences (grant number T32 GM008042), and was formerly employed at Quadrant Health, outside of the submitted work. All other authors declare no competing interests.

Figures

Figure 1:
Figure 1:
Study selection
Figure 2:
Figure 2:. Randomised controlled trials of artificial intelligence in clinical practice across countries and specialties
Norway, France, Sweden, Denmark, Germany, and Slovenia each comprise 2% of the distribution. Canada, Belgium, Ireland, Greece, Latvia, Switzerland, Romania, Bangladesh, Rwanda, Malawi, Viet Nam, and Singapore each comprise 1% of the distribution.
See this image and copyright information in PMC

References

    1. Rajpurkar P, Chen E, Banerjee O, Topol EJ. AI in health and medicine. Nat Med 2022; 28: 31–38. - PubMed
    1. Liu X, Faes L, Kale AU, et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health 2019; 1: e271–97. - PubMed
    1. Rajpurkar P, Lungren MP. The current and future state of AI interpretation of medical images. N Engl J Med 2023; 388: 1981–90. - PubMed
    1. Wu E, Wu K, Daneshjou R, Ouyang D, Ho DE, Zou J. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nat Med 2021; 27: 582–84. - PubMed
    1. Wong A, Otles E, Donnelly JP, et al. External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients. JAMA Intern Med 2021; 181: 1065–70. - PMC - PubMed

Publication types