Developing an Improved Statistical Approach for Survival Estimation in Bone Metastases Management: The Bone Metastases Ensemble Trees for Survival (BMETS) Model

Abstract

Purpose: To determine whether a machine learning approach optimizes survival estimation for patients with symptomatic bone metastases (SBM), we developed the Bone Metastases Ensemble Trees for Survival (BMETS) to predict survival using 27 prognostic covariates. To establish its relative clinical utility, we compared BMETS with 2 simpler Cox regression models used in this setting.

Methods and materials: For 492 bone sites in 397 patients evaluated for palliative radiation therapy (RT) for SBM from January 2007 to January 2013, data for 27 clinical variables were collected. These covariates and the primary outcome of time from consultation to death were used to build BMETS using random survival forests. We then performed Cox regressions as per 2 validated models: Chow's 3-item (C-3) and Westhoff's 2-item (W-2) tools. Model performance was assessed using cross-validation procedures and measured by time-dependent area under the curve (tAUC) for all 3 models. For temporal validation, a separate data set comprised of 104 bone sites treated in 85 patients in 2018 was used to estimate tAUC from BMETS.

Results: Median survival was 6.4 months. Variable importance was greatest for performance status, blood cell counts, recent systemic therapy type, and receipt of concurrent nonbone palliative RT. tAUC at 3, 6, and 12 months was 0.83, 0.81, and 0.81, respectively, suggesting excellent discrimination of BMETS across postconsultation time points. BMETS outperformed simpler models at each time, with respective tAUC at each time of 0.78, 0.76, and 0.74 for the C-3 model and 0.80, 0.78, and 0.77 for the W-2 model. For the temporal validation set, respective tAUC was similarly high at 0.86, 0.82, and 0.78.

Conclusions: For patients with SBM, BMETS improved survival predictions versus simpler traditional models. Model performance was maintained when applied to a temporal validation set. To facilitate clinical use, we developed a web platform for data entry and display of BMETS-predicted survival probabilities.

Fig. 1.

Kaplan-Meier survival estimate (solid line) and 95% confidence interval (dashed line) for the overall group (N = 397).

Fig. 2.

Minimal depth for each covariate within the BMETS model. This value represents the distance between the root node (at position 0) and the node first used to split each covariate, averaged across trees. A lower minimum depth indicates higher prognostic importance for a given variable. Abbreviations: BMETS = Bone Metastases Ensemble Trees for Survival; CC/NFS = central canal and/or neuroforaminal stenosis; KPS = Karnofsky Performance Status; RT = radiation therapy; WBC = white blood cells.

Fig. 3.

Example output for the web platform developed to collect covariate information and display the estimated survival probabilities from time of consultation to death as predicted by the Bone Metastases Ensemble Trees for Survival model. The patient was a 71-year-old black/African American woman with metastatic thyroid cancer who underwent outpatient consultation for a lumbar spine site, with initial cancer diagnosis made 5.25 years ago, most recent systemic therapy of oral (sorafenib) administered ≤1 month ago, no prior surgery to target site, weight loss in the past 6 months, Karnofsky Performance Status of 70, not taking either opiate pain medication or steroids, white blood cell count of 9160, and lymphocyte count of 2390. Imaging showed no definite spinal canal/neuroforaminal compromise, and she had metastasis at other bone sites but no plans for other concurrent palliative radiation therapy. An interactive plot displays the case patient’s predicted survival probabilities after consultation for palliative radiation therapy (orange). For comparison purposes, the curves in the background demonstrate predicted survival probabilities for all other patients with symptomatic bone metastases in the database, displayed from lowest to highest percentile of survival time at 12 months (dark to light blue curves).

Fig. 4.

Comparison of time-dependent area under the curve (tAUC) between prognostic models across survival time points after consultation for palliative radiation therapy. Abbreviations: BMETS = Bone Metastases Ensemble Trees for Survival model; C-3 = Chow’s 3-variable number of risk factors model; W-2 = Westhoff’s 2-variable model.