Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Mar 18;25(1):75.
doi: 10.1186/s12874-025-02528-y.

Using artificial intelligence for systematic review: the example of elicit

Nathan Bernard  1   2 , Yoshimasa Sagawa Jr  3   4 , Nathalie Bier  5   6 , Thomas Lihoreau  3   7   8 , Lionel Pazart  3   4   8 , Thomas Tannou  3   4   5   9
Affiliations

Using artificial intelligence for systematic review: the example of elicit

Nathan Bernard et al. BMC Med Res Methodol. .

Abstract

Background: Artificial intelligence (AI) tools are increasingly being used to assist researchers with various research tasks, particularly in the systematic review process. Elicit is one such tool that can generate a summary of the question asked, setting it apart from other AI tools. The aim of this study is to determine whether AI-assisted research using Elicit adds value to the systematic review process compared to traditional screening methods.

Methods: We compare the results from an umbrella review conducted independently of AI with the results of the AI-based searching using the same criteria. Elicit contribution was assessed based on three criteria: repeatability, reliability and accuracy. For repeatability the search process was repeated three times on Elicit (trial 1, trial 2, trial 3). For accuracy, articles obtained with Elicit were reviewed using the same inclusion criteria as the umbrella review. Reliability was assessed by comparing the number of publications with those without AI-based searches.

Results: The repeatability test found 246,169 results and 172 results for the trials 1, 2, and 3 respectively. Concerning accuracy, 6 articles were included at the conclusion of the selection process. Regarding, revealed 3 common articles, 3 exclusively identified by Elicit and 17 exclusively identified by the AI-independent umbrella review search.

Conclusion: Our findings suggest that AI research assistants, like Elicit, can serve as valuable complementary tools for researchers when designing or writing systematic reviews. However, AI tools have several limitations and should be used with caution. When using AI tools, certain principles must be followed to maintain methodological rigour and integrity. Improving the performance of AI tools such as Elicit and contributing to the development of guidelines for their use during the systematic review process will enhance their effectiveness.

Keywords: Accuracy; Artificial intelligence tools; Reliability; Systematic review writing.

PubMed Disclaimer

Conflict of interest statement

Declarations. Ethical approval and consent to participate: Not Applicable. Consent to publish on part of authors: All authors read and approved the final manuscript. Consent to publish on part of participants: Not Applicable Competing interests: The authors declare no competing interests.

Figures

Fig. 1
Fig. 1
The diagram from Tannou et al. study (left), Elicit (right), and the comparisons at different steps of the systematic review (a, b, c)
See this image and copyright information in PMC

References

    1. Kebede MM, Le Cornet C, Fortner RT. In-depth evaluation of machine learning methods for semi-automating Article screening in a systematic review of mechanistic literature. Res Synth Methods Mars. 2023;14(2):156–72. - PubMed
    1. Blaizot A, Veettil SK, Saidoung P, Moreno-Garcia CF, Wiratunga N, Aceves-Martins M, et al. Using artificial intelligence methods for systematic review in health sciences: A systematic review. Res Synth Methods Mai. 2022;13(3):353–62. - PubMed
    1. Zhang Y, Liang S, Feng Y, Wang Q, Sun F, Chen S, et al. Automation of literature screening using machine learning in medical evidence synthesis: a diagnostic test accuracy systematic review protocol. Syst Rev 15 Janv. 2022;11(1):11. - PMC - PubMed
    1. FAQ| Elicit. [cité 23 mai 2023]. Disponible sur: https://elicit.org/faq#what-is-elicit
    1. Tannou T, Lihoreau T, Couture M, Giroux S, Wang RH, Spalla G, et al. Is research on « smart living environments » based on unobtrusive technologies for older adults going in circles? Evidence from an umbrella review. Ageing Res Rev Févr. 2023;84:101830. - PubMed

LinkOut - more resources