Challenges in meta-analyses with observational studies

Abstract

Objective: Meta-analyses of observational studies are frequently published in the literature, but they are generally considered suboptimal to those involving randomised controlled trials (RCTs) only. This is due to the increased risk of biases that observational studies may entail as well as because of the high heterogeneity that might be present. In this article, we highlight aspects of meta-analyses with observational studies that need more careful consideration in comparison to meta-analyses of RCTs.

Methods: We present an overview of recommendations from the literature with respect to how the different steps of a meta-analysis involving observational studies should be comprehensively conducted. We focus more on issues arising at the step of the quantitative synthesis, in terms of handling heterogeneity and biases. We briefly describe some sophisticated synthesis methods, which may allow for more flexible modelling approaches than common meta-analysis models. We illustrate the issues encountered in the presence of observational studies using an example from mental health, which assesses the risk of myocardial infarction in antipsychotic drug users.

Results: The increased heterogeneity observed among studies challenges the interpretation of the diamond, while the inclusion of short exposure studies may lead to an exaggerated risk for myocardial infarction in this population.

Conclusions: In the presence of observational study designs, prior to synthesis, investigators should carefully consider whether all studies at hand are able to answer the same clinical question. The potential for a quantitative synthesis should be guided through examination of the amount of clinical and methodological heterogeneity and assessment of possible biases.

Figure 1

Forest plot of the main meta-analysis for estimating the risk of myocardial infarction with antipsychotics (AP=antipsycotics, RE=random effects)

Figure 2

Subgroup analysis by study design (AP=antipsycotics, RE=random effects).

Figure 3

Subgroup analysis by type of antipsychotic drug (typical or atypical). AP=antipsycotics, RE=random effects.

Figure 4

Funnel plot of the studies indicating the different study designs with different colours. SCCS, self-controlled case series.