A prospective study comparing the predictions of doctors versus models for treatment outcome of lung cancer patients: a step toward individualized care and shared decision making

Abstract

Background: Decision Support Systems, based on statistical prediction models, have the potential to change the way medicine is being practiced, but their application is currently hampered by the astonishing lack of impact studies. Showing the theoretical benefit of using these models could stimulate conductance of such studies. In addition, it would pave the way for developing more advanced models, based on genomics, proteomics and imaging information, to further improve the performance of the models.

Purpose: In this prospective single-center study, previously developed and validated statistical models were used to predict the two-year survival (2yrS), dyspnea (DPN), and dysphagia (DPH) outcomes for lung cancer patients treated with chemo radiation. These predictions were compared to probabilities provided by doctors and guideline-based recommendations currently used. We hypothesized that model predictions would significantly outperform predictions from doctors.

Materials and methods: Experienced radiation oncologists (ROs) predicted all outcomes at two timepoints: (1) after the first consultation of the patient, and (2) after the radiation treatment plan was made. Differences in the performances of doctors and models were assessed using Area Under the Curve (AUC) analysis.

Results: A total number of 155 patients were included. At timepoint #1 the differences in AUCs between the ROs and the models were 0.15, 0.17, and 0.20 (for 2yrS, DPN, and DPH, respectively), with p-values of 0.02, 0.07, and 0.03. Comparable differences at timepoint #2 were not statistically significant due to the limited number of patients. Comparison to guideline-based recommendations also favored the models.

Conclusion: The models substantially outperformed ROs' predictions and guideline-based recommendations currently used in clinical practice. Identification of risk groups on the basis of the models facilitates individualized treatment, and should be further investigated in clinical impact studies.

Keywords: Lung cancer; Prediction models.

Figure 1

Clinical application of a Decision Support System for stage III NSCLC patients. Shared decision making (SDM), adaptation of treatment (improved radiotherapy planning) and choice of treatment options (either concomitant or sequential chemo radiotherapy and palliative radiotherapy) are integrated in a decision tree based on previously developed and validated prognostic models.

Figure 2

Performance of the models compared to the doctors’ or guideline-based recommendations for (A) death within two years, (B) severe treatment-induced dyspnea, and (C) severe treatment-induced dysphagia. The TNM staging system is often applied as a prognostic tool for survival; for dyspnea and dysphagia, the mean lung radiation dose and maximal esophageal radiation dose were used, respectively. The median value was used to dichotomize the predicted probabilities. For the dyspnea outcome splitting the group was not possible using the gold standard, as the predicted risk was the same for all patients.

Figure 3

Kaplan-Meier Curves for overall survival (months) of 121 NSCLC patients based on (A) clinical TNM stage, (B) predictions of radiation oncologists, and (C) model predictions. P-values of the logrank tests were 0.33, 0.29, and 0.001 respectively.