Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Nov 29;9(12):e23050.
doi: 10.1016/j.heliyon.2023.e23050. eCollection 2023 Dec.

A systematic review and meta-analysis on ChatGPT and its utilization in medical and dental research

Hiroj Bagde  1 , Ashwini Dhopte  2 , Mohammad Khursheed Alam  3   4   5 , Rehana Basri  6
Affiliations

A systematic review and meta-analysis on ChatGPT and its utilization in medical and dental research

Hiroj Bagde et al. Heliyon. .

Abstract

Since its release, ChatGPT has taken the world by storm with its utilization in various fields of life. This review's main goal was to offer a thorough and fact-based evaluation of ChatGPT's potential as a tool for medical and dental research, which could direct subsequent research and influence clinical practices.

Methods: Different online databases were scoured for relevant articles that were in accordance with the study objectives. A team of reviewers was assembled to devise a proper methodological framework for inclusion of articles and meta-analysis.

Results: 11 descriptive studies were considered for this review that evaluated the accuracy of ChatGPT in answering medical queries related to different domains such as systematic reviews, cancer, liver diseases, diagnostic imaging, education, and COVID-19 vaccination. The studies reported different accuracy ranges, from 18.3 % to 100 %, across various datasets and specialties. The meta-analysis showed an odds ratio (OR) of 2.25 and a relative risk (RR) of 1.47 with a 95 % confidence interval (CI), indicating that the accuracy of ChatGPT in providing correct responses was significantly higher compared to the total responses for queries. However, significant heterogeneity was present among the studies, suggesting considerable variability in the effect sizes across the included studies.

Conclusion: The observations indicate that ChatGPT has the ability to provide appropriate solutions to questions in the medical and dentistry areas, but researchers and doctors should cautiously assess its responses because they might not always be dependable. Overall, the importance of this study rests in shedding light on ChatGPT's accuracy in the medical and dentistry fields and emphasizing the need for additional investigation to enhance its performance. © 2017 Elsevier Inc. All rights reserved.

Keywords: Artificial intelligence; ChatGPT; Dentistry; Machine learning; Medicine.

PubMed Disclaimer

Conflict of interest statement

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Figures

Fig. 1
Fig. 1
Article selection framework for the studies included in the review.
Fig. 2
Fig. 2
Bias assessment of the selected papers using the NOS tool.
Fig. 3
Fig. 3
Overall accuracy of ChatGPT in providing correct responses as compared to total responses for queries represented in terms of OR on a forest plot.
Fig. 4
Fig. 4
Overall accuracy of ChatGPT in providing correct responses as compared to total responses for queries represented in terms of RR on a forest plot.
See this image and copyright information in PMC

References

    1. Yu K.H., Beam A.L., Kohane I.S. Artificial intelligence in healthcare. Nat. Biomed. Eng. 2018;2(10):719–731. doi: 10.1038/s41551-018-0305-z. - DOI - PubMed
    1. Xu L., Sanders L., Li K., Chow J.C.L. Chatbot for health care and oncology applications using artificial intelligence and machine learning: systematic review. JMIR Cancer. 2021;7(4) doi: 10.2196/27850. - DOI - PMC - PubMed
    1. Chonde D.B., Pourvaziri A., Williams J., et al. Rad translate: an artificial intelligence– powered intervention for urgent imaging to enhance care equity for patients with limited English proficiency during the COVID-19 pandemic. J. Am. Coll. Radiol. 2021;18(7):1000–1008. doi: 10.1016/j.jacr.2021.01.013. - DOI - PMC - PubMed
    1. Chung J., Kim D., Choi J., et al. Prediction of oxygen requirement in patients with COVID19 using a pre-trained chest radiograph xAI model: efficient development of auditable risk prediction models via a fine-tuning approach. Sci. Rep. 2022;12(1) doi: 10.1038/s41598-022-24721-5. - DOI - PMC - PubMed
    1. Li M.D., Arun N.T., Aggarwal M., et al. Multi-population generalizability of a deep learning based chest radiograph severity score for COVID-19. Medicine (Baltim.) 2022;101(29) doi: 10.1097/MD.0000000000029587. - DOI - PMC - PubMed

LinkOut - more resources