The prediction of distant metastasis risk for male breast cancer patients based on an interpretable machine learning model

Abstract

Objectives: This research was designed to compare the ability of different machine learning (ML) models and nomogram to predict distant metastasis in male breast cancer (MBC) patients and to interpret the optimal ML model by SHapley Additive exPlanations (SHAP) framework.

Methods: Four powerful ML models were developed using data from male breast cancer (MBC) patients in the SEER database between 2010 and 2015 and MBC patients from our hospital between 2010 and 2020. The area under curve (AUC) and Brier score were used to assess the capacity of different models. The Delong test was applied to compare the performance of the models. Univariable and multivariable analysis were conducted using logistic regression.

Results: Of 2351 patients were analyzed; 168 (7.1%) had distant metastasis (M1); 117 (5.0%) had bone metastasis, and 71 (3.0%) had lung metastasis. The median age at diagnosis is 68.0 years old. Most patients did not receive radiotherapy (1723, 73.3%) or chemotherapy (1447, 61.5%). The XGB model was the best ML model for predicting M1 in MBC patients. It showed the largest AUC value in the tenfold cross validation (AUC:0.884; SD:0.02), training (AUC:0.907; 95% CI: 0.899-0.917), testing (AUC:0.827; 95% CI: 0.802-0.857) and external validation (AUC:0.754; 95% CI: 0.739-0.771) sets. It also showed powerful ability in the prediction of bone metastasis (AUC: 0.880, 95% CI: 0.856-0.903 in the training set; AUC: 0.823, 95% CI:0.790-0.848 in the test set; AUC: 0.747, 95% CI: 0.727-0.764 in the external validation set) and lung metastasis (AUC: 0.906, 95% CI: 0.877-0.928 in training set; AUC: 0.859, 95% CI: 0.816-0.891 in the test set; AUC: 0.756, 95% CI: 0.732-0.777 in the external validation set). The AUC value of the XGB model was larger than that of nomogram in the training (0.907 vs 0.802) and external validation (0.754 vs 0.706) sets.

Conclusions: The XGB model is a better predictor of distant metastasis among MBC patients than other ML models and nomogram; furthermore, the XGB model is a powerful model for predicting bone and lung metastasis. Combining with SHAP values, it could help doctors intuitively understand the impact of each variable on outcome.

Keywords: Distant metastasis; Machine learning; Male breast cancer; Nomogram; SEER; XGBoost.

Fig. 1

The flow chart of patients selection and the flow chart for the development, evaluation and explanation of models

Fig. 2

The performance comparison of different ML models. The AUC comparison of different ML models in train set (tenfold cross validation, A). The ROC curves of different ML models in train (B), test (D) and external validation sets (F). The calibration curves of different ML models in train (C), test (E), and external validation sets (G)

Fig. 3

The prediction of bone metastasis based on XGBoost model. The ROC curves of XGBoost model in train (A), test (C) and external validation sets (E). The calibrations of XGBoost model in train (B), test (D) and external validation sets (F)

Fig. 4

The prediction of lung metastasis based on XGBoost model. The ROC curves of XGBoost model in train (A), test (C) and external validation sets (E). The calibrations of XGBoost model in train (B), test (D) and external validation sets (F)

Fig. 5

The XGB model’s interpretation. The importance ranking of the different variables according to the mean (∣SHAP value∣) (A); The importance ranking of different risk factors with stability and interpretation using the optimal model (B). The higher SHAP value of a feature is given, the higher risk of death the patient would have. The red part in feature value represents higher value. A classical sample with distant metastasis (C), and a classical sample without distant metastasis (D)

Fig. 6

Screenshot of the Web APP based on XGBoost model, which is available at https://greenmood.shinyapps.io/male/