Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Aug 25;24(8):e37188.
doi: 10.2196/37188.

Randomized Controlled Trials of Artificial Intelligence in Clinical Practice: Systematic Review

Thomas Y T Lam #  1   2 , Max F K Cheung #  3 , Yasmin L Munro  3 , Kong Meng Lim  3 , Dennis Shung  4 , Joseph J Y Sung  3
Affiliations

Randomized Controlled Trials of Artificial Intelligence in Clinical Practice: Systematic Review

Thomas Y T Lam et al. J Med Internet Res. .

Abstract

Background: The number of artificial intelligence (AI) studies in medicine has exponentially increased recently. However, there is no clear quantification of the clinical benefits of implementing AI-assisted tools in patient care.

Objective: This study aims to systematically review all published randomized controlled trials (RCTs) of AI-assisted tools to characterize their performance in clinical practice.

Methods: CINAHL, Cochrane Central, Embase, MEDLINE, and PubMed were searched to identify relevant RCTs published up to July 2021 and comparing the performance of AI-assisted tools with conventional clinical management without AI assistance. We evaluated the primary end points of each study to determine their clinical relevance. This systematic review was conducted following the updated PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 guidelines.

Results: Among the 11,839 articles retrieved, only 39 (0.33%) RCTs were included. These RCTs were conducted in an approximately equal distribution from North America, Europe, and Asia. AI-assisted tools were implemented in 13 different clinical specialties. Most RCTs were published in the field of gastroenterology, with 15 studies on AI-assisted endoscopy. Most RCTs studied biosignal-based AI-assisted tools, and a minority of RCTs studied AI-assisted tools drawn from clinical data. In 77% (30/39) of the RCTs, AI-assisted interventions outperformed usual clinical care, and clinically relevant outcomes improved with AI-assisted intervention in 70% (21/30) of the studies. Small sample size and single-center design limited the generalizability of these studies.

Conclusions: There is growing evidence supporting the implementation of AI-assisted tools in daily clinical practice; however, the number of available RCTs is limited and heterogeneous. More RCTs of AI-assisted tools integrated into clinical practice are needed to advance the role of AI in medicine.

Trial registration: PROSPERO CRD42021286539; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=286539.

Keywords: artificial intelligence; clinical; clinical informatics; gastroenterology; mobile phone; randomized controlled trial; systematic review.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: None declared.

Figures

Figure 1
Figure 1
PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram.
Figure 2
Figure 2
Number of randomized controlled trials of artificial intelligence–assisted medicine per year.
Figure 3
Figure 3
Distribution of original of publications and specialty.
See this image and copyright information in PMC

References

    1. McCarthy J, Minsky ML, Rochester N, Shannon CE. A proposal for the Dartmouth summer research project on artificial intelligence, August 31, 1955. AI Magazine. 2006 Dec 15;27(4):12. doi: 10.1609/aimag.v27i4.1904. doi: 10.1609/aimag.v27i4.1904. - DOI - DOI
    1. Hamamoto R. Application of artificial intelligence for medical research. Biomolecules. 2021 Jan 12;11(1):90. doi: 10.3390/biom11010090. https://www.mdpi.com/resolver?pii=biom11010090 biom11010090 - DOI - PMC - PubMed
    1. Shung DL, Sung JJ. Challenges of developing artificial intelligence-assisted tools for clinical medicine. J Gastroenterol Hepatol. 2021 Feb;36(2):295–8. doi: 10.1111/jgh.15378. - DOI - PMC - PubMed
    1. Sung JJ, Stewart CL, Freedman B. Artificial intelligence in health care: preparing for the fifth Industrial Revolution. Med J Aust. 2020 Sep;213(6):253–5.e1. doi: 10.5694/mja2.50755. - DOI - PubMed
    1. Brown JM, Campbell JP, Beers A, Chang K, Ostmo S, Chan RV, Dy J, Erdogmus D, Ioannidis S, Kalpathy-Cramer J, Chiang MF, Imaging and Informatics in Retinopathy of Prematurity (i-ROP) Research Consortium Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks. JAMA Ophthalmol. 2018 Jul 01;136(7):803–10. doi: 10.1001/jamaophthalmol.2018.1934. https://europepmc.org/abstract/MED/29801159 2680579 - DOI - PMC - PubMed

Publication types

LinkOut - more resources