Voxel-level radiomics and deep learning for predicting pathologic complete response in esophageal squamous cell carcinoma after neoadjuvant immunotherapy and chemotherapy

Abstract

Background: Accurate prediction of pathologic complete response (pCR) following neoadjuvant immunotherapy combined with chemotherapy (nICT) is crucial for tailoring patient care in esophageal squamous cell carcinoma (ESCC). This study aimed to develop and validate a deep learning model using a novel voxel-level radiomics approach to predict pCR based on preoperative CT images.

Methods: In this multicenter, retrospective study, 741 patients with ESCC who underwent nICT followed by radical esophagectomy were enrolled from three institutions. Patients from one center were divided into a training set (469 patients) and an internal validation set (118 patients) while the data from the other two centers was used as external validation sets (120 and 34 patients, respectively). The deep learning model, Vision-Mamba, integrated voxel-level radiomics feature maps and CT images for pCR prediction. Additionally, other commonly used deep learning models, including 3D-ResNet and Vision Transformer, as well as traditional radiomics methods, were developed for comparison. Model performance was evaluated using accuracy, area under the curve (AUC), sensitivity, specificity, and prognostic stratification capabilities. The SHapley Additive exPlanations analysis was employed to interpret the model's predictions.

Results: The Vision-Mamba model demonstrated robust predictive performance in the training set (accuracy: 0.89, AUC: 0.91, sensitivity: 0.82, specificity: 0.92) and validation sets (accuracy: 0.83-0.91, AUC: 0.83-0.92, sensitivity: 0.73-0.94, specificity: 0.84-1.0). The model outperformed other deep learning models and traditional radiomics methods. The model's ability to stratify patients into high and low-risk groups was validated, showing superior prognostic stratification compared with traditional methods. SHAP provided quantitative and visual model interpretation.

Conclusions: We present a voxel-level radiomics-based deep learning model to predict pCR to neoadjuvant immunotherapy combined with chemotherapy based on pretreatment diagnostic CT images with high accuracy and robustness. This model could provide a promising tool for individualized management of patients with ESCC.

Keywords: Chemotherapy; Esophageal Cancer; Immunotherapy; Neoadjuvant.

Figure 1. Study pipeline. (A) The process involved extracting radiomics feature maps from preoperative CT scans and combining them with CT images to predict pathologic complete response (pCR vs non-pCR) using the Vision-Mamba model. Additionally, the model’s ability to stratify prognosis was explored. (B) Data from three different hospitals were included in the study. The images underwent processing, segmentation, feature extraction, feature selection, and model building. The performance of the model was then validated using independent validation sets. (C) The Vision-Mamba model architecture includes separate convolutional layers for CT images and shared convolutional layers for all radiomics feature maps. After initial processing with these convolutional layers and activation functions, the data is passed through state space model layers. The outputs are then concatenated and fed into fully connected layers to predict pCR or non-pCR. AUC, area under the curve.

Figure 2. Prognostic stratification performance. (A) Kaplan-Meier (KM) curves for overall survival (OS) stratified by actual pathologic complete response (pCR) status in the training set and three independent validation sets (test-set-1, test-set-2, and test-set-3). (B) KM curves for OS stratified by the model’s predicted pCR status. (C) KM curves for OS stratified by the risk scores output by the model, using a median cut-off value of −1.2 from the training set and applying it to the test sets.

Figure 3. Model interpretation and feature importance. (A) Contributions of different input features to the model’s predictions. (B–D) From left to right, each panel shows an original CT image, the cropped tumor region, the SHAP value map, and the overlaid image. The SHAP value maps are overlaid on CT images, with darker red areas indicating regions that contributed more significantly to the model’s predictions. (B) The darker red regions, particularly in the tumor necrotic area (indicated by the arrow), highlight areas with a substantial influence on predicting pathologic complete response. (C) The darker red regions in the tumor edge region indicate significant SHAP values contributing to the model’s predictions. (D) Further visualizations emphasize the importance of enhanced regions in the model’s predictions. SHAP, SHapley Additive exPlanations.