Ensemble deep learning enhanced with self-attention for predicting immunotherapeutic responses to cancers

Abstract

Introduction: Despite the many benefits immunotherapy has brought to patients with different cancers, its clinical applications and improvements are still hindered by drug resistance. Fostering a reliable approach to identifying sufferers who are sensitive to certain immunotherapeutic agents is of great clinical relevance.

Methods: We propose an ELISE (Ensemble Learning for Immunotherapeutic Response Evaluation) pipeline to generate a robust and highly accurate approach to predicting individual responses to immunotherapies. ELISE employed iterative univariable logistic regression to select genetic features of patients, using Monte Carlo Tree Search (MCTS) to tune hyperparameters. In each trial, ELISE selected multiple models for integration based on add or concatenate stacking strategies, including deep neural network, automatic feature interaction learning via self-attentive neural networks, deep factorization machine, compressed interaction network, and linear neural network, then adopted the best trial to generate a final approach. SHapley Additive exPlanations (SHAP) algorithm was applied to interpret ELISE, which was then validated in an independent test set.

Result: Regarding prediction of responses to atezolizumab within esophageal adenocarcinoma (EAC) patients, ELISE demonstrated a superior accuracy (Area Under Curve [AUC] = 100.00%). AC005786.3 (Mean [|SHAP value|] = 0.0097) was distinguished as the most valuable contributor to ELISE output, followed by SNORD3D (0.0092), RN7SKP72 (0.0081), EREG (0.0069), IGHV4-80 (0.0063), and MIR4526 (0.0063). Mechanistically, immunoglobulin complex, immunoglobulin production, adaptive immune response, antigen binding and others, were downregulated in ELISE-neg EAC subtypes and resulted in unfavorable responses. More encouragingly, ELISE could be extended to accurately estimate the responsiveness of various immunotherapeutic agents against other cancers, including PD1/PD-L1 suppressor against metastatic urothelial cancer (AUC = 88.86%), and MAGE-A3 immunotherapy against metastatic melanoma (AUC = 100.00%).

Discussion: This study presented deep insights into integrating ensemble deep learning with self-attention as a mechanism for predicting immunotherapy responses to human cancers, highlighting ELISE as a potential tool to generate reliable approaches to individualized treatment.

Keywords: ELISE; PD1/PD-L1; cancer; deep learning; immunotherapy.

Figure 1

ELISE pipeline. (A) Inputted data. (B) Feature embedding. (C) Neural networks. (D) Hyperparameters tuning.

Figure 2

ELISE applied to EACs. (A) Resultant data of feature selection. (B) Hyperparameter optimization. (C) Loss curves of the best trial. (D, E) presented AUCs of ELISE in the train and test cohort. (F, G) are the calibration plot of ELISE in the train and test cohort, respectively. (H, I) displayed the actual outcomes distribution in the ELISE-neg and ELISE-pos subtypes.

Figure 3

Interpreting ELISE in EACs. (A) SHAP summary plot ranked and presented the top 20 important features. (B, C) exhibited how ELISE makes the global and individual prediction. (D) Dependent plot indicated the affection directions of top 6 features. (E) Many top important features identified by SHAP presented differential expression profiles between ELISE-neg and ELISE-pos subtypes. The symbols ** represents p < 0.01, and *** represents p < 0.001.

Figure 4

Dissecting underlying mechanisms leading to different outcomes. (A) Resultant data of GSEA (BP, CC, MF). (B) GSEA results (signaling pathways). (C) Protein-protein interaction network.

Figure 5

Tumor microenvironments and immune cell infiltration.

Figure 6

ELISE applied in UCs and melanoma. (A) Hyperparameter optimization in UCs. (B) Loss function curves in UCs. (C, D) AUCs of ELISE applied in UCs in the train and test cohort. (E, F) are the calibration plots of ELISE in the train and test cohort, respectively. (G) Hyperparameter optimization in MELANOMAs. (H) Loss function curves in MELANOMAs. (I, J) AMELANOMAs of ELISE applied in MELANOMAs in the train and test cohort. (K, L) are the calibration plots of ELISE in the train and test cohort, respectively.

Figure 7

Interpreting ELISE in UCs. (A) SHAP summary plot ranked and presented the top 20 important features. (B, C) exhibited how ELISE makes the global and individual prediction. (D) Dependent plot indicated the affection directions of top 6 features.