Network Meta-analysis: Users' Guide for Surgeons: Part I - Credibility

Abstract

Conventional meta-analyses quantify the relative effectiveness of two interventions based on direct (that is, head-to-head) evidence typically derived from randomized controlled trials (RCTs). For many medical conditions, however, multiple treatment options exist and not all have been compared directly. This issue limits the utility of traditional synthetic techniques such as meta-analyses, since these approaches can only pool and compare evidence across interventions that have been compared directly by source studies. Network meta-analyses (NMA) use direct and indirect comparisons to quantify the relative effectiveness of three or more treatment options. Interpreting the methodologic quality and results of NMAs may be challenging, as they use complex methods that may be unfamiliar to surgeons; yet for these surgeons to use these studies in their practices, they need to be able to determine whether they can trust the results of NMAs. The first judgment of trust requires an assessment of the credibility of the NMA methodology; the second judgment of trust requires a determination of certainty in effect sizes and directions. In this Users' Guide for Surgeons, Part I, we show the application of evaluation criteria for determining the credibility of a NMA through an example pertinent to clinical orthopaedics. In the subsequent article (Part II), we help readers evaluate the level of certainty NMAs can provide in terms of treatment effect sizes and directions.

Fig. 1

The diagram shows the concept of an open network. In this hypothetical scenario, head-to-head (ie, direct) comparisons of unreamed nailing with reamed nailing and external fixation are available (as indicated by the solid lines). However, no available trials have compared reamed nailing with external fixation (as indicated by the dashed line). The loop enables estimates of effect between reamed nailing and external fixation to be calculated only indirectly through a common comparator (in this case, unreamed nailing).

Fig. 2

The diagram shows the concept of a closed loop network. In this hypothetical scenario, head-to-head trials have compared external fixation with reamed and unreamed nailing (as indicated by the solid lines). Therefore, direct and indirect evidence (derived through the loop involving unreamed nailing) can inform the comparison between external fixation and reamed nailing.

Fig. 3A–D

Four network diagrams in increasing order of complexity are shown. The circles (nodes) represent treatments and the lines represent head-to-head trials (direct evidence). (A) This complex closed loop network shows that as more interventions are added without increasing linkages, the network indirectness increases. (B) This well-connected network shows that if an increase in interventions is complemented with an increase in linkages, then overall indirectness of the network may not increase. (C) A network with a single common comparator is called a “star” network. (D) Complex networks may have many linkages, although typically most studies have concentrated on only a few treatments comparisons. Parts of the network may be well-connected (as indicated by x), while other parts may be poorly connected (as indicated by y).