Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2016 Dec:80:68-76.
doi: 10.1016/j.jclinepi.2016.07.003. Epub 2016 Jul 16.

A threshold analysis assessed the credibility of conclusions from network meta-analysis

Deborah M Caldwell  1 , A E Ades  2 , Sofia Dias  2 , Sarah Watkins  2 , Tianjing Li  3 , Nichole Taske  4 , Bhash Naidoo  4 , Nicky J Welton  2
Affiliations

A threshold analysis assessed the credibility of conclusions from network meta-analysis

Deborah M Caldwell et al. J Clin Epidemiol. 2016 Dec.

Abstract

Objective: To assess the reliability of treatment recommendations based on network meta-analysis (NMA).

Study design and setting: We consider evidence in an NMA to be potentially biased. Taking each pairwise contrast in turn, we use a structured series of threshold analyses to ask: (1) "How large would the bias in this evidence base have to be before it changed our decision?" and (2) "If the decision changed, what is the new recommendation?" We illustrate the method via two NMAs in which a Grading of Recommendations Assessment, Development and Evaluation (GRADE) assessment for NMAs has been implemented: weight loss and osteoporosis.

Results: Four of the weight-loss NMA estimates were assessed as "low" and six as "moderate" quality by GRADE; for osteoporosis, six were "low," nine were "moderate," and 1 was "high." The threshold analysis suggests plausible bias in 3 of 10 estimates in the weight-loss network could have changed the treatment recommendation. For osteoporosis, plausible bias in 6 of 16 estimates could change the recommendation. There was no relation between plausible bias changing a treatment recommendation and the original GRADE assessments.

Conclusions: Reliability judgments on individual NMA contrasts do not help decision makers understand whether a treatment recommendation is reliable. Threshold analysis reveals whether the final recommendation is robust against plausible degrees of bias in the data.

Keywords: Bias; Comparative effectiveness; GRADE; Health technology assessment; Mixed treatment comparison; Quality assessment; Reliability.

PubMed Disclaimer

Figures

Fig. 1
Fig. 1
(A) Network of comparisons as described by Johnston . Edge thickness is proportional to the number of trials contributing to that pairwise contrast. Treatment nodes are not weighted. (B) Osteoporosis network adapted from . Edge thickness is proportional to the number of trials contributing to that pairwise contrast. Treatment nodes are proportional to number of participants.
See this image and copyright information in PMC

References

    1. Dias S., Sutton A.J., Ades A.E., Welton N.J. Evidence synthesis for decision making 2: a generalized linear modeling framework for pairwise and network meta-analysis of randomized controlled trials. Med Decis Making. 2013;33:607–617. - PMC - PubMed
    1. National Institute for Health and Care Excellence . National Institute for Health and Care Excellence; London: 2013. Guide to the methods of technology appraisal 2013. - PubMed
    1. Lu G., Ades A. Combination of direct and indirect evidence in mixed treatment comparisons. Stat In Med. 2004;23:3105–3124. - PubMed
    1. Salanti G., Higgins J.P.T., Ades A.E., Ioannidis J.P.A. Evaluation of networks of randomised trials. Stat Methods Med Res. 2008;17:279–301. - PubMed
    1. Guyatt G.H., Oxman A.D., Schünemann H.J., Tugwell P., Knottnerus A. GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. J Clin Epidemiol. 2011;64:380–382. - PubMed

MeSH terms