Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Sep 18;24(1):210.
doi: 10.1186/s12874-024-02320-4.

An exploration of available methods and tools to improve the efficiency of systematic review production: a scoping review

Lisa Affengruber  1   2 , Miriam M van der Maten  3   4 , Isa Spiero  5 , Barbara Nussbaumer-Streit  6 , Mersiha Mahmić-Kaknjo  7 , Moriah E Ellen  8   9   10 , Käthe Goossen  11 , Lucia Kantorova  12 , Lotty Hooft  4 , Nicoletta Riva  13 , Georgios Poulentzas  14 , Panagiotis Nikolaos Lalagkas  14 , Anabela G Silva  15 , Michele Sassano  16   17 , Raluca Sfetcu  18   19 , María E Marqués  20 , Tereza Friessova  21 , Eduard Baladia  20 , Angelo Maria Pezzullo  16 , Patricia Martinez  20   22 , Gerald Gartlehner  6   23 , René Spijker  4   24
Affiliations

An exploration of available methods and tools to improve the efficiency of systematic review production: a scoping review

Lisa Affengruber et al. BMC Med Res Methodol. .

Abstract

Background: Systematic reviews (SRs) are time-consuming and labor-intensive to perform. With the growing number of scientific publications, the SR development process becomes even more laborious. This is problematic because timely SR evidence is essential for decision-making in evidence-based healthcare and policymaking. Numerous methods and tools that accelerate SR development have recently emerged. To date, no scoping review has been conducted to provide a comprehensive summary of methods and ready-to-use tools to improve efficiency in SR production.

Objective: To present an overview of primary studies that evaluated the use of ready-to-use applications of tools or review methods to improve efficiency in the review process.

Methods: We conducted a scoping review. An information specialist performed a systematic literature search in four databases, supplemented with citation-based and grey literature searching. We included studies reporting the performance of methods and ready-to-use tools for improving efficiency when producing or updating a SR in the health field. We performed dual, independent title and abstract screening, full-text selection, and data extraction. The results were analyzed descriptively and presented narratively.

Results: We included 103 studies: 51 studies reported on methods, 54 studies on tools, and 2 studies reported on both methods and tools to make SR production more efficient. A total of 72 studies evaluated the validity (n = 69) or usability (n = 3) of one method (n = 33) or tool (n = 39), and 31 studies performed comparative analyses of different methods (n = 15) or tools (n = 16). 20 studies conducted prospective evaluations in real-time workflows. Most studies evaluated methods or tools that aimed at screening titles and abstracts (n = 42) and literature searching (n = 24), while for other steps of the SR process, only a few studies were found. Regarding the outcomes included, most studies reported on validity outcomes (n = 84), while outcomes such as impact on results (n = 23), time-saving (n = 24), usability (n = 13), and cost-saving (n = 3) were less often evaluated.

Conclusion: For title and abstract screening and literature searching, various evaluated methods and tools are available that aim at improving the efficiency of SR production. However, only few studies have addressed the influence of these methods and tools in real-world workflows. Few studies exist that evaluate methods or tools supporting the remaining tasks. Additionally, while validity outcomes are frequently reported, there is a lack of evaluation regarding other outcomes.

Keywords: Automation tools; Evidence synthesis; Method; Rapid review; Scoping review; Systematic review.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

Fig. 1
Fig. 1
Steps of the SR process. *added to Tsafnat et al.’s list [25]. ** added based on Nussbaumer-Streit et al. [14]
Fig. 2
Fig. 2
PRISMA flowchart
Fig. 3
Fig. 3
The number of identified evaluation studies per review step
Fig. 4
Fig. 4
Outcomes reported in the included studies
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Oliver S, Dickson K, Bangpan M. Systematic reviews: making them policy relevant. A briefing for policy makers and systematic reviewers. London: EPPI-Centre, Social Science Research Unit, UCL Institute of Education, University College London; 2015.
    1. Borah R, Brown AW, Capers PL, Kaiser KA. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open. 2017;7(2):e012545. - PMC - PubMed
    1. Donnelly CA, Boyd I, Campbell P, Craig C, Vallance P, Walport M, Whitty CJM, Woods E, Wormald C. Four principles to make evidence synthesis more useful for policy. Nature. 2018;558(7710):361–4. - PubMed
    1. Clayton GL, Smith IL, Higgins JPT, Mihaylova B, Thorpe B, Cicero R, Lokuge K, Forman JR, Tierney JF, White IR, et al. The INVEST project: investigating the use of evidence synthesis in the design and analysis of clinical trials. Trials. 2017;18(1):219. - PMC - PubMed
    1. Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, McInerney P, Godfrey CM, Khalil H. Updated methodological guidance for the conduct of scoping reviews. JBI Evidence Synthesis. 2020;18(10). - PubMed

Publication types

LinkOut - more resources