Predicting diagnosis and survival of bone metastasis in breast cancer using machine learning

Abstract

This study aimed at establishing more accurate predictive models based on novel machine learning algorithms, with the overarching goal of providing clinicians with effective decision-making assistance. We retrospectively analyzed the breast cancer patients recorded in the Surveillance, Epidemiology, and End Results (SEER) database from 2010 to 2016. Multivariable logistic regression analyses were used to identify risk factors for bone metastases in breast cancer, whereas Cox proportional hazards regression analyses were used to identify prognostic factors for breast cancer with bone metastasis (BCBM). Based on the identified risk and prognostic factors, we developed diagnostic and prognostic models that incorporate six machine learning classifiers. We then used the area under the receiver operating characteristic (ROC) curve (AUC), learning curve, precision curve, calibration plot, and decision curve analysis to evaluate performance of the machine learning models. Univariable and multivariable logistic regression analyses showed that bone metastases were significantly associated with age, race, sex, grade, T stage, N stage, surgery, radiotherapy, chemotherapy, tumor size, brain metastasis, liver metastasis, lung metastasis, breast subtype, and PR. Univariate and multivariate Cox regression analyses revealed that age, race, marital status, grade, surgery, radiotherapy, chemotherapy, brain metastasis, liver metastasis, lung metastasis, breast subtype, ER, and PR were closely associated with the prognosis of BCBM. Among the six machine learning models, the XGBoost algorithm predicted the most accurate results (Diagnostic model AUC = 0.98; Prognostic model AUC = 0.88). According to the Shapley additive explanations (SHAP), the most critical feature of the diagnostic model was surgery, followed by N stage. Interestingly, surgery was also the most critical feature of prognostic model, followed by liver metastasis. Based on the XGBoost algorithm, we could effectively predict the diagnosis and survival of bone metastasis in breast cancer and provide targeted references for the treatment of BCBM patients.

Figure 1

Flow chart of patient screening.

Figure 2

Learning curves of models with training data. (A) XGBoost; (B) Random Forest; (C) Decision Trees; (D) Extra Trees; (E) Gaussian NB; (F) Logistic regression.

Figure 3

ROC curves of diagnostic models developed by training cohort (A) and validation cohort (B); PR curves of models developed by training cohort (C) and validation cohort (D); calibration curves of models developed by training cohort (E) and validation cohort (F).

Figure 4

Ten-fold cross-validation results of the six machine learning models in the training group.

Figure 5

The ROC curves of prognostic models based on machine learning in training set (A) and validation set (B). The decision curves of prognostic models based on machine learning in training cohort (C) and validation cohort (D).

Figure 6

Prediction performance of seven models.

Figure 7

Feature importance ranking by SHAP values in diagnostic model based on the XGBoost algorithm. (A) The features are sorted according to the sum of the SHAP values of all patients, and SHAP values are used to represent the distribution of the influence of each feature on the output of the XGBoost model. Red indicates that the value of the feature is higher, whereas blue indicates that the value of the feature is lower. The X-axis represents the effect of SHAP values on the output of the model. The higher the value of X-axis, the greater the likelihood of delayed mitigation. (B) The standard bar chart is drawn and sorted using the average absolute value of each feature shape value in the XGBoost model.

Figure 8

Feature importance ranking by SHAP values in prognostic model based on XGBoost algorithm. (A) The features are sorted according to the sum of the SHAP values of all patients, and SHAP values are used to represent the distribution of the influence of each feature on the output of the XGBoost model. Red indicates that the value of the feature is higher, whereas blue indicates that the value of the feature is lower. The X-axis represents the effect of SHAP values on the output of the model. The higher the value of X-axis, the greater the likelihood of delayed mitigation. (B) The standard bar chart is drawn and sorted using the average absolute value of each feature shape value in the XGBoost model.