Impact of reporting bias in network meta-analysis of antidepressant placebo-controlled trials

Abstract

Background: Indirect comparisons of competing treatments by network meta-analysis (NMA) are increasingly in use. Reporting bias has received little attention in this context. We aimed to assess the impact of such bias in NMAs.

Methods: We used data from 74 FDA-registered placebo-controlled trials of 12 antidepressants and their 51 matching publications. For each dataset, NMA was used to estimate the effect sizes for 66 possible pair-wise comparisons of these drugs, the probabilities of being the best drug and ranking the drugs. To assess the impact of reporting bias, we compared the NMA results for the 51 published trials and those for the 74 FDA-registered trials. To assess how reporting bias affecting only one drug may affect the ranking of all drugs, we performed 12 different NMAs for hypothetical analysis. For each of these NMAs, we used published data for one drug and FDA data for the 11 other drugs.

Findings: Pair-wise effect sizes for drugs derived from the NMA of published data and those from the NMA of FDA data differed in absolute value by at least 100% in 30 of 66 pair-wise comparisons (45%). Depending on the dataset used, the top 3 agents differed, in composition and order. When reporting bias hypothetically affected only one drug, the affected drug ranked first in 5 of the 12 NMAs but second (n = 2), fourth (n = 1) or eighth (n = 2) in the NMA of the complete FDA network.

Conclusions: In this particular network, reporting bias biased NMA-based estimates of treatments efficacy and modified ranking. The reporting bias effect in NMAs may differ from that in classical meta-analyses in that reporting bias affecting only one drug may affect the ranking of all drugs.

Figure 1. Star-shaped networks of comparisons of data from 74 US Food and Drug Administration (FDA)-registered trials of 12 antidepressants and their 51 related publications.

The central node represents the placebo, and each leaf node represents an antidepressant agent. Each node diameter is proportional to the number of patients who received the antidepressant agent; each connecting line width is proportional to the number of trials that addressed the comparison.

Figure 2. Scatterplot of estimates of relative efficacy for 66 pair-wise comparisons of the 12 antidepressant agents with one another derived from network meta-analyses of data from 74 FDA-registered trials and their 51 trial publications.

Data are effect sizes. Positive effect sizes indicate that drug A has higher efficacy than drug B. The two areas above the uppermost dotted line (labeled +100%) and below the lowest dotted line (labeled −100%) correspond to cases in which an effect size derived from the network meta-analysis of the 51 published trials differed in absolute value from that derived from the network meta-analysis of the 74 FDA-registered trials by at least 100%. The two areas between the 2 upper dotted lines (labeled +50%) and between the 2 lower dotted lines (labeled −50%) correspond to cases in which an effect size derived from the network meta-analysis of the 51 published trials differed in absolute value from that derived from the network meta-analysis of the 74 FDA-registered trials by at least 50%. Red-colored points refer to cases in which agent B was superior to agent A by one network meta-analysis and A was superior to B by the other network meta-analysis.

Figure 3. Probabilities that each antidepressant drug is the best according to network meta-analyses of data from 74 FDA-registered trials or 51 published trials with published effect sizes.

For instance, for mirtazapine, the probability of being the best was 7.3% and 30.2% according to network-meta-analysis of the 74 FDA-registered trials and 51 published trials with published effect sizes, respectively. Drugs for which the probability of being the best was <5% for both published and FDA data are not labeled (blue area).

Figure 4. Probabilities of being the best among competing antidepressant agents when reporting bias affects one specific agent.

The first stacked bar at the left corresponds to the network meta-analysis free of reporting biases (ie, with the data from the 74 FDA-registered trials). The other stacked bars correspond to the 12 network meta-analyses in which reporting bias hypothetically affects one specific agent in turn. For instance, for mirtazapine, we used the 6 published trials (out of 10 FDA-registered trials), with published effect sizes, and data from the 64 FDA-registered trials for the other 11 agents, which resulted in an incomplete FDA network of 70 trials; the probability of mirtazapine being the best was 80.6% with data from the incomplete FDA network and 7.3% with data from the 74 FDA-registered trials. For the sake of clarity, we presented in each analysis the 3 drugs with the 3 highest probabilities of being the best among competing antidepressant agents. Bup: Bupropion; Cit: Citalopram; Dul : Duloxetine ; Esc: Escitalopram; Flu: Fluoxetine; Mir: Mirtazapine ; Nef: Nefazodone ; Par: Paroxetine; Par CR: Paroxetine CR; Ser: Sertraline; Ven: Venlafaxine; VenXR: Venlafaxine XR.