CINeMA: Software for semiautomated assessment of the confidence in the results of network meta-analysis

Abstract

Network meta-analysis (NMA) compares several interventions that are linked in a network of comparative studies and estimates the relative treatment effects between all treatments, using both direct and indirect evidence. NMA is increasingly used for decision making in health care, however, a user-friendly system to evaluate the confidence that can be placed in the results of NMA is currently lacking. This paper is a tutorial describing the Confidence In Network Meta-Analysis (CINeMA) web application, which is based on the framework developed by Salanti et al (2014, PLOS One, 9, e99682) and refined by Nikolakopoulou et al (2019, bioRxiv). Six domains that affect the level of confidence in the NMA results are considered: (a) within-study bias, (b) reporting bias, (c) indirectness, (d) imprecision, (e) heterogeneity, and (f) incoherence. CINeMA is freely available and open-source and no login is required. In the configuration step users upload their data, produce network plots and define the analysis and effect measure. The dataset should include assessments of study-level risk of bias and judgments on indirectness. CINeMA calls the netmeta routine in R to estimate relative effects and heterogeneity. Users are then guided through a systematic evaluation of the six domains. In this way reviewers assess the level of concerns for each relative treatment effect from NMA as giving rise to "no concerns," "some concerns," or "major concerns" in each of the six domains, which are graphically summarized on the report page for all effect estimates. Finally, judgments across the domains are summarized into a single confidence rating ("high," "moderate," "low," or "very low"). In conclusion, the user-friendly web-based CINeMA platform provides a transparent framework to evaluate evidence from systematic reviews with multiple interventions.

Figure 1

Network plot for the network meta‐analysis of antihypertensive drugs and diabetes incidence using four different sizing and coloring combinations. Green, yellow, and red colors refer to low, moderate, and high risk of bias or indirectness. The plot can be downloaded as a .png file by clicking on “Save Plot” in “Configuration.” ACE, angiotensin‐converting‐enzyme inhibitors; ARB, angiotensin‐receptor blockers; BBlocker, Beta Blocker; CCB, calcium‐channel blocker

Figure 2

Risk of bias bar chart for the network meta‐analysis of antihypertensive drugs and diabetes incidence. Each bar represents the evidence for a relative treatment effect. White vertical lines separate colored areas which refer to the contribution of each study. Each bar shows the percentage contribution from studies judged to be at low (green), moderate (yellow), and high (red) risk of bias. The plot can be downloaded as a .png file by clicking on “Save Chart” in “Within‐study bias.” ACE, angiotensin‐converting‐enzyme inhibitors; ARB, angiotensin‐receptor blockers; CCB, calcium‐channel blocker

Figure 3

Boxes showing the judgments for within‐study bias for all relative effects in the network meta‐analysis of antihypertensive drugs and diabetes incidence. ACE, angiotensin‐converting‐enzyme inhibitors; ARB, angiotensin‐receptor blockers; CCB, calcium‐channel blocker

Figure 4

Illustration of rules to assess imprecision (a), heterogeneity (b), and incoherence (c) in CINeMA. We assume several fictional scenarios for the odds ratio from NMA comparing interventions X and Y. The clinically important effects were set at 0.8 and 1.25 ( = 1/0.8). The gray areas represent values that favor neither of the competing interventions. The shaded interval represents the interval between the null effect and clinically important size of effect. Black horizontal lines indicate confidence intervals and red extensions indicate prediction intervals of NMA relative treatment effects. Dotted lines represent direct and dashed lines represent indirect confidence intervals. Judgments are the same for cases symmetrical to those illustrated. NMA, network meta‐analysis

Figure 5

Boxes showing the information for judging heterogeneity for the relative effect of beta blockers versus placebo in the network meta‐analysis of antihypertensive drugs and diabetes incidence

Figure 6

Boxes showing the judgments for incoherence for all relative effects in the network meta‐analysis of antihypertensive drugs and diabetes incidence. ACE, angiotensin‐converting‐enzyme inhibitors; ARB, angiotensin‐receptor blockers; CCB, calcium‐channel blocker

Figure 7

Final output of CINeMA for the network of antihypertensive drugs and incidence of diabetes. The table shows the level of concern for each of the six domains for each comparison and can be downloaded as a .csv file by clicking on “Download Report” in “Report.” ACE, angiotensin‐converting‐enzyme inhibitors; ARB, angiotensin‐receptor blockers; CCB, calcium‐channel blocker