Development and Validation of an Interpretable Machine Learning Prediction Model for Total Pathological Complete Response after Neoadjuvant Chemotherapy in Locally Advanced Breast Cancer: Multicenter Retrospective Analysis

Abstract

Objective: This study aims to develop an interpretable machine learning (ML) model to accurately predict the probability of achieving total pathological complete response (tpCR) in patients with locally advanced breast cancer (LABC) following neoadjuvant chemotherapy (NAC). Methods: This multi-center retrospective study included pre-NAC clinical pathology data from 698 LABC patients. Post-operative pathological outcomes divided patients into tpCR and non-tpCR groups. Data from 586 patients at Shanghai Ruijin Hospital were randomly assigned to a training set (80%) and a test set (20%). In comparison, data from our hospital's remaining 112 patients were used for external validation. Variable selection was performed using the Least Absolute Shrinkage and Selection Operator (LASSO) regression analysis. Predictive models were constructed using six ML algorithms: decision trees, K-nearest neighbors (KNN), support vector machine, light gradient boosting machine, and extreme gradient boosting. Model efficacy was assessed through various metrics, including receiver operating characteristic (ROC) curves, precision-recall (PR) curves, confusion matrices, calibration plots, and decision curve analysis (DCA). The best-performing model was selected by comparing the performance of different algorithms. Moreover, variable relevance was ranked using the SHapley Additive exPlanations (SHAP) technique to improve the interpretability of the model and solve the "black box" problem. Results: A total of 191 patients (32.59%) achieved tpCR following NAC. Through LASSO regression analysis, five variables were identified as predictive factors for model construction, including tumor size, Ki-67, molecular subtype, targeted therapy, and chemotherapy regimen. The KNN model outperformed the other five classifier algorithms, achieving area under the curve (AUC) values of 0.847 (95% CI: 0.809-0.883) in the training set, 0.763 (95% CI: 0.670-0.856) in the test set, and 0.665 (95% CI: 0.555-0.776) in the external validation set. DCA demonstrated that the KNN model yielded the highest net advantage through a wide range of threshold probabilities in both the training and test sets. Furthermore, the analysis of the KNN model utilizing SHAP technology demonstrated that targeted therapy is the most crucial factor in predicting tpCR. Conclusion: An ML prediction model using clinical and pathological data collected before NAC was developed and verified. This model accurately predicted the probability of achieving a tpCR in patients with LABC after receiving NAC. SHAP technology enhanced the interpretability of the model and assisted in clinical decision-making and therapy optimization.

Keywords: Locally advanced breast cancer; Machine learning; Neoadjuvant chemotherapy; Pathological complete response; Predictive model; SHapley additive exPlanations.

Figure 1

The flowchart of the study.

Figure 2

Representative images of ER, PR, and HER-2 by immunohistochemical staining. ER (A) and PR (B) negative expression in the nucleus. ER (C) and PR (D) showed strong positive in the nucleus; E.HER-2(0) (no staining or incomplete and faint/barely perceptible membrane staining in ≤10% of tumor cells); F HER-2(+) (incomplete and faint/barely perceptible membrane staining in >10% of tumor cells); G HER-2(++) (weak/moderate complete membrane staining in>10% of tumor cells or complete and intense membrane staining in ≤10% of tumor cells); H HER-2(+++) (complete and intense membrane staining in>10% of tumor cells); Magnification (X20), Scale bars = 100 µm.

Figure 3

(A and B) LASSO regression model. A delineates selecting the most appropriate regularization parameter, λ, employing a ten-fold cross-validation approach within the LASSO regression framework. B showcases a coefficient profile plot, which is constructed based on the sequence of log (λ) values, providing insights into the behavior of the model's coefficients across different values of λ. C and D display the receiver operating characteristic (ROC) curves for six distinct models within the training and test sets, respectively. E and F present these models' precision-recall (PR) curves, comparing their performance in both the training and test sets.

Figure 4

Calibration plots of six models in the training set (A) and the test set (B).

Figure 5

The DCA was conducted for six models in both the training set (A) and test set (B) using different threshold probabilities; in the DCA plots, the bottom gray line represents the scenario where no patients achieved tpCR following NAC, while the black diagonal line represents the scenario where all patients achieved tpCR after NAC. The x-axis of DCA represents the threshold probability, and the y-axis represents the net benefit after subtracting the disadvantages. Theoretically, the further the DCA curve is from these two extreme lines, the higher the net clinical benefit of the model. The external validation set contains the AUC (C) and calibration plot (D).

Figure 6

A: Aggregate SHAP values for categorical variables; B: Feature importance plot; in the feature importance plot, each predictive variable corresponds to a line segment, the length of which indicates the weight of the variable's impact on tpCR. A longer line signifies a higher weight, reflecting the variable's importance in predicting tpCR. C: A patient's waterfall plot demonstrating tpCR; D: Individual waterfall plot for a patient who did not achieve tpCR. E [f(x)] represents the baseline prediction probability; f(x) denotes the model's final prediction probability for a given input.