A Systematic Review of the Literature Demonstrates Some Errors in the Use of Decision Curve Analysis but Generally Correct Interpretation of Findings

Abstract

Background. Decision curve analysis (DCA) is a widely used methodology in clinical research studies. Purpose. We performed a literature review to identify common errors in the application of DCA and provide practical suggestions for appropriate use of DCA. Data Sources. We first conducted an informal literature review and identified 6 errors found in some DCAs. We then used Google Scholar to conduct a systematic review of studies applying DCA to evaluate a predictive model, marker, or test. Data Extraction. We used a standard data collection form to collect data for each reviewed article. Data Synthesis. Each article was assessed according to the 6 predefined criteria for a proper analysis, reporting, and interpretation of DCA. Overall, 50 articles were included in the review: 54% did not select an appropriate range of probability thresholds for the x-axis of the DCA, with a similar proportion (50%) failing to present smoothed curves. Among studies with internal validation of a predictive model and correction for overfit, 61% did not clearly report whether the DCA had also been corrected. However, almost all studies correctly interpreted the DCA, used a correct outcome (92% for both), and clearly reported the clinical decision at issue (81%). Limitations. A comprehensive assessment of all DCAs was not performed. However, such a strategy would not influence the main findings. Conclusions. Despite some common errors in the application of DCA, our finding that almost all studies correctly interpreted the DCA results demonstrates that it is a clear and intuitive method to assess clinical utility.

Keywords: decision curve analysis; prediction; quality.

Figure 1:

Flow charts showing inclusion and exclusion criteria of articles for the review. Articles were selected in reverse chronological order.