Predicting the Tumor Microenvironment Composition and Immunotherapy Response in Non-Small Cell Lung Cancer from Digital Histopathology Images

Abstract

Immune checkpoint inhibitors (ICI) have become integral to treatment of non-small cell lung cancer (NSCLC). However, reliable biomarkers predictive of immunotherapy efficacy are limited. Here, we introduce HistoTME, a novel weakly supervised deep learning approach to infer the tumor microenvironment (TME) composition directly from histopathology images of NSCLC patients. We show that HistoTME accurately predicts the expression of 30 distinct cell type-specific molecular signatures directly from whole slide images, achieving an average Pearson correlation of 0.5 with the ground truth on independent tumor cohorts. Furthermore, we find that HistoTME-predicted microenvironment signatures and their underlying interactions improve prognostication of lung cancer patients receiving immunotherapy, achieving an AUROC of 0.75[95% CI: 0.61-0.88] for predicting treatment responses following first-line ICI treatment, utilizing an external clinical cohort of 652 patients. Collectively, HistoTME presents an effective approach for interrogating the TME and predicting ICI response, complementing PD-L1 expression, and bringing us closer to personalized immuno-oncology.

Figure 1:

(A-B) Overview of Study Design and pipeline of HistoTME. Each Whole Slide Image is tessellated into smaller tiles and preprocessed by a pretrained digital pathology foundation model to extract meaningful tile embeddings. The tile embeddings generated by the foundation model are then provided as input to an attention-based multiple instance learning (AB-MIL) module followed by a multi-layer perceptron head (MLP), which learns to predict expression levels to 30 tumor microenvironment-related molecular signatures. Overall, to develop HistoTME we experiment with three open-source foundation models - CTransPath, RetCCL, and UNI^,, - and two configurations of AB-MIL: single task AB-MIL, where the predictions of each signature are optimized separately, and multi-task AB-MIL, where predictions of functionally related signatures are jointly optimized. The signature prediction performance of each foundation model coupled with each configuration of AB-MIL is shown on held out CPTAC validation data in Supplementary Figure 3. Overall, the UNI foundation model + multitask AB-MIL produces the most accurate predictions and is hence chosen as the final version of HistoTME (C) Pearson correlations between the ground truth expression levels of each patient derived from bulk transcriptomics and predicted expression levels of each patient derived from the final version of HistoTME (UNI+multi-task AB-MIL) on the held out CPTAC validation cohort. (D) Pearson and Spearman correlations between the cell type abundance of each patient, defined as the number of marker positive cells per mm² from immunohistochemistry (IHC) slides, and the predicted cell type-specific signature expression levels of each patient derived from final version of HistoTME (UNI+multitask AB-MIL) is shown on the external SUNY Upstate test cohort. Error bars represent the 95% confidence intervals. Cell type abundances were estimated from whole slide immunohistochemistry images using QuPath v0.5.0 cell detection and classification algorithms with default parameter settings. TME = tumor microenvironment; LUAD = lung adenocarcinoma; LUSC = lung squamous cell carcinoma; MLP = multilayer perceptron

Figure 2

(A): Overview of the computational pipeline to classify patients into distinct clusters based on their H&E-predicted TME composition. H&E stained digitized tumor samples from TCGA+CPTAC are processed by HistoTME and subsequently clustered into two clusters based on partition around medoid (PAM) clustering and a Random Forest classification model that is trained on cluster membership data (B) 3D PCA plot visualizing the two distinct clusters of TCGA + CPTAC NSCLC patients: Immune Inflamed and Immune Desert, based on their HistoTME-inferred TME profiles. (C) Heatmap depicting the H&E-predicted TME composition and clinical attributes of NSCLC patients from the SUNY cohort. Patients were classified into Immune Inflamed cluster or Immune Desert cluster using a two class classification model (Random Forest) trained on TCGA+CPTAC data. (D) Random forrest-derived feature importance rankings of TME signatures driving the distinction between the Immune Inflamed and Immune Desert cluster.(E) Immunohistochemistry-derived T cell, B cell and Macrophage abundances, defined as number of marker positive cells per mm², in cases belonging to the immune-inflamed or immune desert cluster. Cell type abundances were quantified from whole slide immunohistochemistry images using QuPath v0.5.0 positive cell detection and quantification pipline. Statistical significance between groups was determined by non-parametric Wilox rank sum test (***: p-value < 0.001, ****: p-value < 0.0001)

Figure 3

(A-B): Low magnification view of a primary NSCLC tumor resection sample and its predicted TME signature profile. (C-E) Matched whole slide immunohistochemistry images of the same tumor sample dual stained for CD4 (brown)+CD8 (magenta), CD3 (brown)+CD20(magenta), and P40 (brown)+CD163 (magenta) markers respectively. (F) HistoTME generated attention maps for each attention head. Below each whole slide attention map are 4 high magnification image tiles (50×50 μm) randomly sampled from high attention areas. Supplementary Figure 6 shows another related example along with higher magnification image tiles randomly sampled from high attention areas.

Figure 4

(A-B): Low magnification view of a metastatic NSCLC tumor resection sample and its predicted TME signature profile. (C-E) Matched whole slide immunohistochemistry images of the same tumor sample dual stained for CD4 (brown)+CD8 (magenta)markers, CD3 (brown)+CD20(magenta) markers, and P40 (brown)+CD163 (magenta) markers respectively. (F) HistoTME generated attention maps for each attention head. Below each whole slide attention map are 4 representative high magnification image tiles (50×50 μm) sampled from high attention areas. Supplementary Figure 7 shows another related example along with higher magnification image tiles randomly sampled from high attention areas.

Figure 5:

Association between HistoTME-based TME classification and overall survival outcomes of SUNY NSCLC patients treated with first-line anti-PD1/PD-L1 therapy (first-line IO patients). (A) Kaplan Meier plot depicting overall survival−defined as time from date of diagnosis to date of death −of patients that received first-line anti-PD1/PD-L1 treatment (B) Kaplan Meier plot depicting overall survival of SUNY patients that received first-line anti-PD1/PD-L1 therapy stratified by PD-L1 IHC expression (C-E) Kaplan Meier plots depicting overall survival of first-line patients in PD-L1 negative (TPS < 1%), PD-L1 low (TPS = 1–49%) and PD-L1 high (>= 50%), cases. Significance of survival differences between distinct subgroups of patients was determined by the log-rank test.

Figure 6:

(A) Model development for response prediction: 1) HistoTME predictions are engineered into new features by taking pairwise sums, differences, products, and quotients. 2) random forest feature selection. 3) XGBoost trained for response prediction. (B) Feature network pairwise interactions of 18 selected features. Arrow endpoints denote the signature subtracted or divided from the signature at the start point. (C) Test set receiver operating characteristic (ROC) curve of the model trained on engineered features or TME signatures alone. Optimal cut point shown based on the Youden index. Kaplan Meier plot depicting overall survival of the test set stratified by AI response prediction for (D) all patients that received anti-PD1/PD-L1 treatment and (E) first-line immunotherapy (IO)-treated patients. (F) Shapley additive explanation (SHAP) summary plot ordered by SHAP importance.