Assessing key assumptions of network meta-analysis: a review of methods

Abstract

Background: Homogeneity and consistency assumptions underlie network meta-analysis (NMA). Methods exist to assess the assumptions but they are rarely and poorly applied. We review and illustrate methods to assess homogeneity and consistency.

Methods: Eligible articles focussed on indirect comparison or NMA methodology. Articles were sought by hand-searching and scanning references (March 2013). Assumption assessment methods described in the articles were reviewed, and applied to compare anti-malarial drugs.

Results: 116 articles were included. Methods to assess homogeneity were: comparing characteristics across trials; comparing trial-specific treatment effects; using hypothesis tests or statistical measures; applying fixed-effect and random-effects pair-wise meta-analysis; and investigating treatment effect-modifiers. Methods to assess consistency were: comparing characteristics; investigating treatment effect-modifiers; comparing outcome measurements in the referent group; node-splitting; inconsistency modelling; hypothesis tests; back transformation; multidimensional scaling; a two-stage approach; and a graph-theoretical method. For the malaria example, heterogeneity existed for some comparisons that was unexplained by investigating treatment effect-modifiers. Inconsistency was detected using node-splitting and inconsistency modelling. It was unclear whether the covariates explained the inconsistency.

Conclusions: Presently, we advocate applying existing assessment methods collectively to gain the best understanding possible regarding whether assumptions are reasonable. In our example, consistency was questionable; therefore the NMA results may be unreliable.

Keywords: consistency, transitivity; mixed treatment comparison; multiple treatments meta-analysis; network meta-analysis.