Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Meta-Analysis
. 2024 Feb 21;13(1):70.
doi: 10.1186/s13643-024-02464-w.

Synthesis methods used to combine observational studies and randomised trials in published meta-analyses

Cherifa Cheurfa  1   2 , Sofia Tsokani  3 , Katerina-Maria Kontouli  3 , Isabelle Boutron  4   5   6 , Anna Chaimani  4   6
Affiliations
Meta-Analysis

Synthesis methods used to combine observational studies and randomised trials in published meta-analyses

Cherifa Cheurfa et al. Syst Rev. .

Abstract

Background: This study examined the synthesis methods used in meta-analyses pooling data from observational studies (OSs) and randomised controlled trials (RCTs) from various medical disciplines.

Methods: We searched Medline via PubMed to identify reports of systematic reviews of interventions, including and pooling data from RCTs and OSs published in 110 high-impact factor general and specialised journals between 2015 and 2019. Screening and data extraction were performed in duplicate. To describe the synthesis methods used in the meta-analyses, we considered the first meta-analysis presented in each article.

Results: Overall, 132 reports were identified with a median number of included studies of 14 [9-26]. The median number of OSs was 6.5 [3-12] and that of RCTs was 3 [1-6]. The effect estimates recorded from OSs (i.e., adjusted or unadjusted) were not specified in 82% (n = 108) of the meta-analyses. An inverse-variance common-effect model was used in 2% (n = 3) of the meta-analyses, a random-effects model was used in 55% (n = 73), and both models were used in 40% (n = 53). A Poisson regression model was used in 1 meta-analysis, and 2 meta-analyses did not report the model they used. The mean total weight of OSs in the studied meta-analyses was 57.3% (standard deviation, ± 30.3%). Only 44 (33%) meta-analyses reported results stratified by study design. Of them, the results between OSs and RCTs had a consistent direction of effect in 70% (n = 31). Study design was explored as a potential source of heterogeneity in 79% of the meta-analyses, and confounding factors were investigated in only 10% (n = 13). Publication bias was assessed in 70% (n = 92) of the meta-analyses. Tau-square was reported in 32 meta-analyses with a median of 0.07 [0-0.30].

Conclusion: The inclusion of OSs in a meta-analysis on interventions could provide useful information. However, considerations of several methodological and conceptual aspects of OSs, that are required to avoid misleading findings, were often absent or insufficiently reported in our sample.

Keywords: Comparative effectiveness, heterogeneous designs; Data synthesis; Non-randomised studies.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interests.

Figures

Fig. 1
Fig. 1
Flow chart of study selection
Fig. 2
Fig. 2
Confounding factors explored in the meta-analyses
See this image and copyright information in PMC

References

    1. Ziff OJ, Lane DA, Samra M, Griffith M, Kirchhof P, Lip GYH, et al. Safety and efficacy of digoxin: systematic review and meta-analysis of observational and controlled trial data. BMJ. 2015;351:h4451. doi: 10.1136/bmj.h4451. - DOI - PMC - PubMed
    1. Metelli S, Chaimani A. Challenges in meta-analyses with observational studies. Evid Based Ment Health. 2020;23:83–87. doi: 10.1136/ebmental-2019-300129. - DOI - PMC - PubMed
    1. Shrier I, Boivin J-F, Steele RJ, Platt RW, Furlan A, Kakuma R, et al. Should meta-analyses of interventions include observational studies in addition to randomized controlled trials? A critical examination of underlying principles. Am J Epidemiol. 2007;166:1203–1209. doi: 10.1093/aje/kwm189. - DOI - PubMed
    1. Abraham NS, Byrne CJ, Young JM, Solomon MJ. Meta-analysis of well-designed nonrandomized comparative studies of surgical procedures is as good as randomized controlled trials. J Clin Epidemiol. 2010;63:238–245. doi: 10.1016/j.jclinepi.2009.04.005. - DOI - PubMed
    1. Tinetti ME. The gap between clinical trials and the real world: extrapolating treatment effects from younger to older adults. JAMA Intern Med. 2014;174:397–398. doi: 10.1001/jamainternmed.2013.13283. - DOI - PubMed

Publication types

LinkOut - more resources