. 2024 Jun 25:26:e56780.

doi: 10.2196/56780.

Potential Roles of Large Language Models in the Production of Systematic Reviews and Meta-Analyses

Xufei Luo^{1

2

3

4

5}, Fengxian Chen⁶, Di Zhu⁷, Ling Wang⁷, Zijun Wang^{1

2

3

4

5}, Hui Liu^{1

2

3

4

5}, Meng Lyu⁷, Ye Wang⁷, Qi Wang^{8

9}, Yaolong Chen^{1

2

3

4

5}

Affiliations

¹ Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou, China.
² World Health Organization Collaboration Center for Guideline Implementation and Knowledge Translation, Lanzhou, China.
³ Institute of Health Data Science, Lanzhou University, Lanzhou, China.
⁴ Key Laboratory of Evidence Based Medicine and Knowledge Translation of Gansu Province, Lanzhou University, Lanzhou, China.
⁵ Research Unit of Evidence-Based Evaluation and Guidelines, Chinese Academy of Medical Sciences (2021RU017), School of Basic Medical Sciences, Lanzhou University, Lanzhou, China.
⁶ School of Information Science & Engineering, Lanzhou University, Lanzhou, China.
⁷ School of Public Health, Lanzhou University, Lanzhou, China.
⁸ Department of Health Research Methods, Evidence and Impact, Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada.
⁹ McMaster Health Forum, McMaster University, Hamilton, ON, Canada.

PMID: 38819655
PMCID: PMC11234072
DOI: 10.2196/56780

Potential Roles of Large Language Models in the Production of Systematic Reviews and Meta-Analyses

Xufei Luo et al. J Med Internet Res. 2024.

. 2024 Jun 25:26:e56780.

doi: 10.2196/56780.

Authors

Xufei Luo^{1

2

3

4

5}, Fengxian Chen⁶, Di Zhu⁷, Ling Wang⁷, Zijun Wang^{1

2

3

4

5}, Hui Liu^{1

2

3

4

5}, Meng Lyu⁷, Ye Wang⁷, Qi Wang^{8

9}, Yaolong Chen^{1

2

3

4

5}

Affiliations

¹ Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou, China.
² World Health Organization Collaboration Center for Guideline Implementation and Knowledge Translation, Lanzhou, China.
³ Institute of Health Data Science, Lanzhou University, Lanzhou, China.
⁴ Key Laboratory of Evidence Based Medicine and Knowledge Translation of Gansu Province, Lanzhou University, Lanzhou, China.
⁵ Research Unit of Evidence-Based Evaluation and Guidelines, Chinese Academy of Medical Sciences (2021RU017), School of Basic Medical Sciences, Lanzhou University, Lanzhou, China.
⁶ School of Information Science & Engineering, Lanzhou University, Lanzhou, China.
⁷ School of Public Health, Lanzhou University, Lanzhou, China.
⁸ Department of Health Research Methods, Evidence and Impact, Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada.
⁹ McMaster Health Forum, McMaster University, Hamilton, ON, Canada.

PMID: 38819655
PMCID: PMC11234072
DOI: 10.2196/56780

Abstract

Large language models (LLMs) such as ChatGPT have become widely applied in the field of medical research. In the process of conducting systematic reviews, similar tools can be used to expedite various steps, including defining clinical questions, performing the literature search, document screening, information extraction, and language refinement, thereby conserving resources and enhancing efficiency. However, when using LLMs, attention should be paid to transparent reporting, distinguishing between genuine and false content, and avoiding academic misconduct. In this viewpoint, we highlight the potential roles of LLMs in the creation of systematic reviews and meta-analyses, elucidating their advantages, limitations, and future research directions, aiming to provide insights and guidance for authors planning systematic reviews and meta-analyses.

Keywords: ChatGPT; chatbot; large language model; meta-analysis; systematic review.

©Xufei Luo, Fengxian Chen, Di Zhu, Ling Wang, Zijun Wang, Hui Liu, Meng Lyu, Ye Wang, Qi Wang, Yaolong Chen. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 25.06.2024.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: None declared.

Figures

**Figure 1**
The process of conducting a systematic review and meta-analysis.

See this image and copyright information in PMC

Cited by

Evaluating the Quality of Guidelines Using the AGREE II Tool by a Large Language Model vs Human Appraisers.
Wang B, Luo X, Zhang J, Shi Q, Lai H, Liu H, Yao Y, Ge L, Liu J, Li H, Ma Y, Zhang L, Bian Z, Florez ID, Chen Y, Estill J. Wang B, et al. JAMA Netw Open. 2025 May 1;8(5):e2512621. doi: 10.1001/jamanetworkopen.2025.12621. JAMA Netw Open. 2025. PMID: 40434777 Free PMC article.
Accelerating clinical evidence synthesis with large language models.
Wang Z, Cao L, Danek B, Jin Q, Lu Z, Sun J. Wang Z, et al. NPJ Digit Med. 2025 Aug 8;8(1):509. doi: 10.1038/s41746-025-01840-7. NPJ Digit Med. 2025. PMID: 40775042 Free PMC article.
Large Language Models in Medicine: Applications, Challenges, and Future Directions.
Yu E, Chu X, Zhang W, Meng X, Yang Y, Ji X, Wu C. Yu E, et al. Int J Med Sci. 2025 May 31;22(11):2792-2801. doi: 10.7150/ijms.111780. eCollection 2025. Int J Med Sci. 2025. PMID: 40520893 Free PMC article. Review.
High-performance automated abstract screening with large language model ensembles.
Sanghera R, Thirunavukarasu AJ, El Khoury M, O'Logbon J, Chen Y, Watt A, Mahmood M, Butt H, Nishimura G, Soltan AAS. Sanghera R, et al. J Am Med Inform Assoc. 2025 May 1;32(5):893-904. doi: 10.1093/jamia/ocaf050. J Am Med Inform Assoc. 2025. PMID: 40119675 Free PMC article.
GPT-3.5 Turbo and GPT-4 Turbo in Title and Abstract Screening for Systematic Reviews.
Oami T, Okada Y, Nakada TA. Oami T, et al. JMIR Med Inform. 2025 Mar 12;13:e64682. doi: 10.2196/64682. JMIR Med Inform. 2025. PMID: 40073422 Free PMC article.

See all "Cited by" articles

References

1. Jahan N, Naveed S, Zeshan M, Tahir MA. How to conduct a systematic review: a narrative literature review. Cureus. 2016 Nov 04;8(11):e864. doi: 10.7759/cureus.864. https://europepmc.org/abstract/MED/27924252 - DOI - PMC - PubMed
1. Wallace SS, Barak G, Truong G, Parker MW. Hierarchy of evidence within the medical literature. Hosp Pediatr. 2022 Aug 01;12(8):745–750. doi: 10.1542/hpeds.2022-006690.188605 - DOI - PubMed
1. Institute of Medicine. Board on Health Care Services. Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. Graham R, Mancher M, Miller Wolman D, Greenfield S, Steinberg E. Clinical Practice Guidelines We Can Trust. Washington, DC: National Academies Press; 2011. - PubMed
1. Bystranowski P, Janik B, Próchnicki M, Skórska P. Anchoring effect in legal decision-making: a meta-analysis. Law Hum Behav. 2021 Feb;45(1):1–23. doi: 10.1037/lhb0000438.2021-26899-001 - DOI - PubMed
1. Geyskens I, Krishnan R, Steenkamp JEM, Cunha PV. A review and evaluation of meta-analysis practices in management research. J Management. 2008 Feb 05;35(2):393–419. doi: 10.1177/0149206308328501. - DOI

MeSH terms

Actions
Actions
Actions
Actions

LinkOut - more resources

Full Text Sources
- JMIR Publications
- PubMed Central

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Potential Roles of Large Language Models in the Production of Systematic Reviews and Meta-Analyses

Affiliations

Potential Roles of Large Language Models in the Production of Systematic Reviews and Meta-Analyses

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

Similar articles

Cited by

References

MeSH terms

LinkOut - more resources

Full Text Sources

Abstract

Conflict of interest statement

Figures

Similar articles

Cited by

References

MeSH terms

Related information

LinkOut - more resources

Full Text Sources