Methods for evidence synthesis in the case of very few studies

Abstract

In systematic reviews, meta-analyses are routinely applied to summarize the results of the relevant studies for a specific research question. If one can assume that in all studies the same true effect is estimated, the application of a meta-analysis with common effect (commonly referred to as fixed-effect meta-analysis) is adequate. If between-study heterogeneity is expected to be present, the method of choice is a meta-analysis with random effects. The widely used DerSimonian and Laird method for meta-analyses with random effects has been criticized due to its unfavorable statistical properties, especially in the case of very few studies. A working group of the Cochrane Collaboration recommended the use of the Knapp-Hartung method for meta-analyses with random effects. However, as heterogeneity cannot be reliably estimated if only very few studies are available, the Knapp-Hartung method, while correctly accounting for the corresponding uncertainty, has very low power. Our aim is to summarize possible methods to perform meaningful evidence syntheses in the situation with only very few (ie, 2-4) studies. Some general recommendations are provided on which method should be used when. Our recommendations are based on the existing literature on methods for meta-analysis with very few studies and consensus of the authors. The recommendations are illustrated by 2 examples coming from dossier assessments of the Institute for Quality and Efficiency in Health Care.

Keywords: common effect; evidence synthesis; fixed effects; meta-analysis; random effects; very few studies.

Figure 1

Results of the common‐effects model with inverse‐variance method (CE IV) and the random‐effects model with the DerSimonian‐Laird method (DSL), the Knapp‐Hartung method (KH), and Bayesian method using half‐normal priors for τ with scales 0.5 (B‐HN(0.5)) and 1.0 (B‐HN(1.0)) for the belatacept example [Colour figure can be viewed at http://wileyonlinelibrary.com]

Figure 2

Results of the common‐effects model with inverse‐variance method (CE IV), the random‐effects model with the DerSimonian‐Laird method (DSL) and the Knapp‐Hartung method (KH), the beta‐binomial model (BBIN), and Bayesian method using half‐normal priors for τ with scales 0.5 (B‐HN(0.5)) and 1.0 (B‐HN(1.0)) for the sipuleucel‐T example [Colour figure can be viewed at http://wileyonlinelibrary.com]