Adjustment for reporting bias in network meta-analysis of antidepressant trials

Abstract

Background: Network meta-analysis (NMA), a generalization of conventional MA, allows for assessing the relative effectiveness of multiple interventions. Reporting bias is a major threat to the validity of MA and NMA. Numerous methods are available to assess the robustness of MA results to reporting bias. We aimed to extend such methods to NMA.

Methods: We introduced 2 adjustment models for Bayesian NMA. First, we extended a meta-regression model that allows the effect size to depend on its standard error. Second, we used a selection model that estimates the propensity of trial results being published and in which trials with lower propensity are weighted up in the NMA model. Both models rely on the assumption that biases are exchangeable across the network. We applied the models to 2 networks of placebo-controlled trials of 12 antidepressants, with 74 trials in the US Food and Drug Administration (FDA) database but only 51 with published results. NMA and adjustment models were used to estimate the effects of the 12 drugs relative to placebo, the 66 effect sizes for all possible pair-wise comparisons between drugs, probabilities of being the best drug and ranking of drugs. We compared the results from the 2 adjustment models applied to published data and NMAs of published data and NMAs of FDA data, considered as representing the totality of the data.

Results: Both adjustment models showed reduced estimated effects for the 12 drugs relative to the placebo as compared with NMA of published data. Pair-wise effect sizes between drugs, probabilities of being the best drug and ranking of drugs were modified. Estimated drug effects relative to the placebo from both adjustment models were corrected (i.e., similar to those from NMA of FDA data) for some drugs but not others, which resulted in differences in pair-wise effect sizes between drugs and ranking.

Conclusions: In this case study, adjustment models showed that NMA of published data was not robust to reporting bias and provided estimates closer to that of NMA of FDA data, although not optimal. The validity of such methods depends on the number of trials in the network and the assumption that conventional MAs in the network share a common mean bias mechanism.

Figure 1

Contour-enhanced funnel plots forthe antidepressant trials withpublished results. Each funnel plot is the scatter plot of the treatment effect estimates from individual trials against the associated standard errors; the vertical solid line represents the pooled estimate. In the absence of reporting bias, we might expect a symmetrical funnel plot. We may find the funnel plot is not symmetrical, ie does not resemble an inverted funnel, which may be due to reporting bias, however there are other possible sources of asymmetry. The contour lines represent perceived milestones of statistical significance (long dash p = 0.1; dash p = 0.05; short dash p = 0.01). If studies seem to be missing in areas of non-significance then asymmetry may be due to reporting bias rather than other factors.

Figure 2

Difference plots of estimatesof pair-wise comparisons ofthe 12 antidepressant agentsand placebo: regression modelof published data vs.standard network meta-analysis (NMA)model of published data(left panel); selection modelsof published vs. standardNMA model of publisheddata (right panel). The x-axes show the estimates from the standard NMA model applied to published data, the y-axes show the differences between the estimates from the adjustment (regression or selection) model of published data and the estimates from the standard NMA model of published data. Black dots are the 12 estimated drug effects relative to placebo; white dots are the 66 possible pair-wise comparisons between the 12 drugs.

Figure 3

Probabilities that each antidepressantdrug is the bestaccording to standard NMAof FDA data, regressionmodel, selection model orstandard NMA model ofpublished data.

Figure 4

Cumulative ranking probability plotsfor the 12 antidepressantagents from the standardNMA model applied toFDA data (bold solidline) and published data(bold dotted line) andfrom the 2 adjustmentmodels applied to publisheddata (regression model inplain dashed line andselection model in plaindouble-dashed line). On each plot, the x-axis shows possible ranks from r = 1 up to r = 13 and the y-axis shows the cumulative probabilities that the corresponding treatment is among the top r treatments. The closer the curve is to the upper left corner, the better the treatment. The surface under the cumulative ranking line is 1 when a treatment is the best and 0 when a treatment is the worst. FDA: standard NMA model applied to FDA data (bold plain line); Pub.: standard NMA model applied to published data (bold dash line); Reg.: regression model applied to published data (dash line); Sln.: selection model applied to published data (long-dash short-dash line).

Figure 5

Difference plots of estimatesof pair-wise comparisons ofthe 12 antidepressant agentsand placebo: standard NMAmodel of published datavs. standard NMA modelof FDA data (upperpanel); regression model ofpublished data vs. standardNMA model of FDAdata (bottom left panel);selection model of publishedvs. standard NMA modelof FDA data (bottomright panel). The x-axes show the estimates from the standard NMA model applied to FDA data, the y-axes show the differences between the estimates from the adjustment (regression or selection) model of published data and the estimates from the standard NMA model of FDA data. Black dots are the 12 estimated drug effects relative to placebo; white dots are the 66 possible pair-wise comparisons between the 12 drugs.