Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2018 Dec;74(6):796-804.
doi: 10.1016/j.eururo.2018.08.038. Epub 2018 Sep 19.

Reporting and Interpreting Decision Curve Analysis: A Guide for Investigators

Ben Van Calster  1 , Laure Wynants  2 , Jan F M Verbeek  3 , Jan Y Verbakel  4 , Evangelia Christodoulou  2 , Andrew J Vickers  5 , Monique J Roobol  3 , Ewout W Steyerberg  6
Affiliations
Review

Reporting and Interpreting Decision Curve Analysis: A Guide for Investigators

Ben Van Calster et al. Eur Urol. 2018 Dec.

Abstract

Context: Urologists regularly develop clinical risk prediction models to support clinical decisions. In contrast to traditional performance measures, decision curve analysis (DCA) can assess the utility of models for decision making. DCA plots net benefit (NB) at a range of clinically reasonable risk thresholds.

Objective: To provide recommendations on interpreting and reporting DCA when evaluating prediction models.

Evidence acquisition: We informally reviewed the urological literature to determine investigators' understanding of DCA. To illustrate, we use data from 3616 patients to develop risk models for high-grade prostate cancer (n=313, 9%) to decide who should undergo a biopsy. The baseline model includes prostate-specific antigen and digital rectal examination; the extended model adds two predictors based on transrectal ultrasound (TRUS).

Evidence synthesis: We explain risk thresholds, NB, default strategies (treat all, treat no one), and test tradeoff. To use DCA, first determine whether a model is superior to all other strategies across the range of reasonable risk thresholds. If so, that model appears to improve decisions irrespective of threshold. Second, consider if there are important extra costs to using the model. If so, obtain the test tradeoff to check whether the increase in NB versus the best other strategy is worth the additional cost. In our case study, addition of TRUS improved NB by 0.0114, equivalent to 1.1 more detected high-grade prostate cancers per 100 patients. Hence, adding TRUS would be worthwhile if we accept subjecting 88 patients to TRUS to find one additional high-grade prostate cancer or, alternatively, subjecting 10 patients to TRUS to avoid one unnecessary biopsy.

Conclusions: The proposed guidelines can help researchers understand DCA and improve application and reporting.

Patient summary: Decision curve analysis can identify risk models that can help us make better clinical decisions. We illustrate appropriate reporting and interpretation of decision curve analysis.

Keywords: Clinical utility; Decision curve analysis; Net benefit; Risk prediction models; Risk threshold; Test tradeoff.

PubMed Disclaimer

Conflict of interest statement

Financial disclosures: Ben Van Calster certifies that all conflicts of interest, including specific financial interests and relationships and affiliations relevant to the subject matter or materials discussed in the manuscript (eg, employment/affiliation, grants or funding, consultancies, honoraria, stock ownership or options, expert testimony, royalties, or patents filed, received, or pending), are the following: None.

Figures

Fig. 1 –
Fig. 1 –
Decision curves for the default strategies and for the baseline and extended models.
Fig. 2 –
Fig. 2 –
Hypothetical decision curves illustrating several possible scenarios.
See this image and copyright information in PMC

References

    1. Steyerberg EW. Clinical prediction models: a practical approach to development, validation, and updating. New York, NY: Springer; 2009.
    1. Shariat SF, Kattan MW, Vickers AJ, Karakiewicz PI, Scardino PT. Critical review of prostate cancer predictive tools. Future Oncol 2009;5:1555–84. - PMC - PubMed
    1. Vickers AJ, Cronin AM. Everything you always wanted to know about evaluating prediction models (but were too afraid to ask). Urology 2010;76:1298–301. - PMC - PubMed
    1. Van Calster B, Nieboer D, Vergouwe Y, De Cock B, Pencina MJ, Steyerberg EW. A calibration hierarchy for risk models was defined: from utopia to empirical data. J Clin Epidemiol 2016;74:167–76. - PubMed
    1. Van Calster B, Vickers AJ. Calibration of risk prediction models: impact on decision-analytic performance. Med Decis Making 2015;35:162–9. - PubMed

Publication types

MeSH terms