Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Jun:158:44-52.
doi: 10.1016/j.jclinepi.2023.02.008. Epub 2023 Feb 21.

Empirical evaluation of the methods used in systematic reviews including observational studies and randomized trials

Cherifa Cheurfa  1 , Sofia Tsokani  2 , Katerina-Maria Kontouli  2 , Isabelle Boutron  3 , Anna Chaimani  4
Affiliations

Empirical evaluation of the methods used in systematic reviews including observational studies and randomized trials

Cherifa Cheurfa et al. J Clin Epidemiol. 2023 Jun.

Abstract

Objectives: To examine the methodological characteristics of systematic reviews and meta-analyses including observational studies (OSs) and randomized controlled trials (RCTs), in various medical disciplines.

Study design and setting: We searched Medline via PubMed to identify systematic reviews of interventions including RCTs and OSs published in 110 journals from 2015 to 2019. We extracted in duplicate general and methodological characteristics of the systematic review.

Results: We identified 402 systematic reviews. Only 39% (n = 160) of them reported the availability of a pre-established protocol. A rationale for including observational data in the systematic review was clearly reported in 25% (n = 102) of the systematic reviews. Thirty two percent (n = 130) of the reviews reported a search strategy intending to identify published and unpublished data for RCTs and OSs. The risk of bias of the individual studies was assessed in 89% (n = 359) of the systematic reviews. In 74% (n = 266) it was assessed for both RCTs and OSs; 180 (50%) used different tools. Information about confounding factors was reported in only 11% of systematic reviews and the type of effect estimates (crude or adjusted) used was specified in only 22% of the systematic reviews. Among the 385 systematic reviews that performed data synthesis, only 132 (33%) pooled OSs and RCTs in the same meta-analysis.

Conclusion: Including OSs in systematic reviews of interventions could provide useful information but such an approach could also be misleading; thus, several methodological details are needed to ensure appropriate handling of OS and valid results. Our study revealed, although, that substantial methodological information is missing in reports published in high-impact factor general and specialty journals.

Keywords: Comparative effectiveness research; Heterogeneous designs; Intervention; Meta-analyses; Non randomized studies; Systematic review.

PubMed Disclaimer

LinkOut - more resources