CINeMA: An approach for assessing confidence in the results of a network meta-analysis

Abstract

Background: The evaluation of the credibility of results from a meta-analysis has become an important part of the evidence synthesis process. We present a methodological framework to evaluate confidence in the results from network meta-analyses, Confidence in Network Meta-Analysis (CINeMA), when multiple interventions are compared.

Methodology: CINeMA considers 6 domains: (i) within-study bias, (ii) reporting bias, (iii) indirectness, (iv) imprecision, (v) heterogeneity, and (vi) incoherence. Key to judgments about within-study bias and indirectness is the percentage contribution matrix, which shows how much information each study contributes to the results from network meta-analysis. The contribution matrix can easily be computed using a freely available web application. In evaluating imprecision, heterogeneity, and incoherence, we consider the impact of these components of variability in forming clinical decisions.

Conclusions: Via 3 examples, we show that CINeMA improves transparency and avoids the selective use of evidence when forming judgments, thus limiting subjectivity in the process. CINeMA is easy to apply even in large and complicated networks.

Fig 1. Network plots of 2 network meta-analyses.

(A) Network of randomised controlled trials comparing non-invasive diagnostic strategies for the detection of coronary artery disease in patients presenting with symptoms suggestive of acute coronary syndrome. The width of the edges is proportional to the number of patients randomised in each comparison. The colours of edges and nodes refer to the risk of bias: low (green), moderate (yellow), and high (red). (B) Network of randomised controlled trials comparing statins with respect to adverse effects. CCTA, coronary computed tomography angiography; CMR, cardiovascular magnetic resonance; ECG, electrocardiogram; echo, echocardiography; SPECT-MPI, single photon emission computed tomography–myocardial perfusion imaging.

Fig 2. Risk of bias bar chart for the comparison of non-invasive diagnostic strategies for the detection of coronary artery disease.

Each bar represents a relative treatment effect estimated from the network shown in Fig 1A. White vertical lines separate the percentage contribution of different studies. Each bar shows the percentage contribution from studies judged to be at low (green), moderate (yellow), and high (red) risk of bias.

Fig 3. CINeMA rules to assess imprecision, heterogeneity, and incoherence of network treatment effects.

Black lines show confidence intervals, and red lines prediction intervals. The shaded grey area represents values that favour neither of the competing interventions: the range of equivalence, from 0.8 to 1.25. The hatched area shows the interval between the no effect line and a clinically important effect in the opposite direction to the observed effect. For incoherence, dashed lines represent direct effects, and dotted lines indirect effects.

Fig 4. Odds ratios for treatment discontinuation due to adverse effects from the network of statins with their 95% CIs and their 95% PrIs.

Black lines represent 95% CIs, and red lines represent 95% PrIs. The range of equivalence is from 0.95 to 1.05. CI, confidence interval; PrI, prediction interval; vs, versus.