Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers

Abstract

Background: In the last decade, network meta-analysis of randomized controlled trials has been introduced as an extension of pairwise meta-analysis. The advantage of network meta-analysis over standard pairwise meta-analysis is that it facilitates indirect comparisons of multiple interventions that have not been studied in a head-to-head fashion. Although assumptions underlying pairwise meta-analyses are well understood, those concerning network meta-analyses are perceived to be more complex and prone to misinterpretation.

Discussion: In this paper, we aim to provide a basic explanation when network meta-analysis is as valid as pairwise meta-analysis. We focus on the primary role of effect modifiers, which are study and patient characteristics associated with treatment effects. Because network meta-analysis includes different trials comparing different interventions, the distribution of effect modifiers cannot only vary across studies for a particular comparison (as with standard pairwise meta-analysis, causing heterogeneity), but also between comparisons (causing inconsistency). If there is an imbalance in the distribution of effect modifiers between different types of direct comparisons, the related indirect comparisons will be biased. If it can be assumed that this is not the case, network meta-analysis is as valid as pairwise meta-analysis.

Summary: The validity of network meta-analysis is based on the underlying assumption that there is no imbalance in the distribution of effect modifiers across the different types of direct treatment comparisons, regardless of the structure of the evidence network.

Figure 1

Treatment vs study effect in a randomized controlled trial, and the role of effect modifiers.

Figure 2

Standard pairwise meta-analysis of four AB trials (comparing treatment B relative to A). (a) No differences in the effect modifier ‘disease severity’ across studies and therefore no between-study heterogeneity. (b) Extreme differences in the effect modifier across studies and therefore between-study heterogeneity. (Not useful to pool the studies). (c) Differences in the distribution of the effect modifier across studies and therefore between-study heterogeneity. Given the inclusion of both severe and moderate patients in each of the studies there is also within-study heterogeneity, but this cannot be observed without access to subgroup or individual patient-level data.

Figure 3

Valid network meta-analysis of AB trials (comparing treatment B relative to A) and AC trials (comparing treatment C relative to A). (a) Forest plot of four AB studies and four AC studies. There are no differences in the effect modifier across studies within comparisons and no imbalance in the distribution of the effect modifier between comparisons. Hence, there is no heterogeneity, and no bias in the indirect comparison estimate of C vs B. (b) Forest plot of four AB studies and four AC studies. There are differences in the effect modifier across studies within comparisons, but no imbalance in the distribution of the effect modifier between comparisons. Hence, there is heterogeneity, but no bias in the indirect comparison estimate of C vs B.

Figure 4

Biased network meta-analysis of AB trials (comparing treatment B relative to A) and AC trials (comparing treatment C relative to A). (a) Forest plot of four AB studies and four AC studies. There are no differences in the effect modifier across studies within comparisons, but an imbalance in the distribution of the effect modifier between comparisons. Hence, there is no between-study heterogeneity, but a biased indirect comparison estimate of C vs B. (b) Forest plot of four AB studies and four AC studies. There are differences in the distribution of the effect modifier across studies within comparisons, as well as an imbalance in the distribution of the effect modifier between comparisons. Hence, there is between-study heterogeneity, and a biased indirect comparison estimate of C vs B.

Figure 5

Given a certain distribution of effect modifiers across studies, grouping of studies for the analysis has an impact on heterogeneity and inconsistency. (a) Pooled results of all trials as obtained with a pairwise meta-analysis results in great amount of heterogeneity. (b) Network meta-analysis with grouping of studies according to type of comparison (AD, AE, or AF) results in inconsistency in the absence of heterogeneity. (c) Network meta-analysis with grouping of studies by treatment class (AD and AE vs AF) results in absence of inconsistency but presence of between-study heterogeneity (across studies 1 to 4).