Artificial Intelligence Algorithm Predicts Response to Immune Checkpoint Inhibitors

Abstract

Purpose: Cancer treatment has been revolutionized by immune checkpoint inhibitors (ICI). However, a subset of patients do not respond and/or they experience significant adverse events. Attempts to integrate reliable biomarkers of ICI response as part of standard care have been hampered by limited generalizability. We previously reported our supervised machine learning (ML) model in a retrospective cohort of metastatic melanoma.

Experimental design: In this study, we expanded our testing to include larger cohorts of patients with melanoma accrued at several sites, including patients enrolled in clinical trials in both adjuvant and metastatic settings. We examined pretreatment hematoxylin and eosin slides from 639 patients with stage III/IV melanoma treated with ICIs [anti-cytotoxic T-lymphocyte-associated protein 4 (n = 212), anti-programmed death 1 (n = 271), or the combination (n = 156)]. We tested the generalizability of our supervised ML algorithm to predict response to ICIs in the metastatic melanoma cohort and then developed a self-supervised ML model to identify the histologic morphologies associated with patients' survival following ICI use in adjuvant and metastatic melanoma cohorts.

Results: We predicted the response to ICI treatment with an AUC of 0.72. The deep convolutional neural network classified patients into high and low risk based on their likelihood of progression-free survival (P < 0.0001). We uncovered a novel association of specific histomorphologic tumor features-epithelioid histology and a low tumor-stroma ratio-with survival following ICI treatment.

Conclusions: Our data support the generalizability of our developed ML algorithm in predicting response to ICI treatment in patients with metastatic unresectable melanoma. We also showed, for the first time, tumor features associated with patients' overall survival.

Figure 1.

Validation of a self-supervised classifier on the CheckMate-067 cohort. A, Tiled slides. B, Mask of manually annotated tumors performed by our pathologist. C, Mask of the manually annotated nontumor regions performed by our pathologist. D, Heatmap and (E) segmented map generated by the neural network classifier. F, Performance of the segmentation classifier. G, Performance of the response (red) and logistic regression classifier with clinical data (ICI) only (blue) or with the DCNN (black) incorporating clinical data (ICI). H, Multivariable classifier combining the response classifier prediction and clinical data to predict progression-free survival (PFS) probability. AI, artificial intelligence; CI, confidence interval.

Figure 2.

UMAP and PAGA representations of the SSL. A, A UMAP plot revealing 52 separate HPCs (each dot represents a tile). B, The same UMAP plot as in A but colored by survival status (dead or alive). C, PAGA representation of B with the proportion of tiles from each group in each HPC represented by nodes (circles) and edges connecting most similar nodes. D–F, The PAGA graph displays the cluster representation for pathologist annotations, including tumor–stroma ratio, lymphocyte infiltration, and tumor morphology (for visual simplicity, only nodes of PAGA are represented from now on). NA, not applicable.

Figure 3.

Concordance index between individual HPCs and OS. Mean C-index values across the 4-fold cross-validation shown for each HPC as a table (A–C) and on the PAGA graph (D–F) and computed regardless of the treatment (A and D) for anti–CTLA-4–treated patients only (B and E) and anti–PD-1–treated patients only (C and F). For values shown in tables (A–C), the first row shows results for all datasets used to train the self-supervised network, the last row shows results for all datasets combined, and the intermediate rows show results for each dataset individually. For the combined datasets (first and last rows), we show only the most statistically relevant c-indexes (P value < 0.05), whereas the remaining lines show the c-index split for each institution (corresponding P values shown in Supplementary Fig. S7). Values less than 0.5 (yellow) indicate that a higher percentage of the HPC favors longer survival (good outcome), whereas those greater than 0.5 (purple) indicate that enrichment in features from those HPCs favor shorter survival (poor outcome). Similar information is shown as a PAGA graph (D–F), with a color scheme proportional to the c-index value and size reflecting the P value (larger circles have P values < 0.05, smaller ones have >0.1, and intermediate ones have 0.05 < P value < 0.1).

Figure 4.

The table and PAGA graph display clusters that are connected to tumor response outcomes, such as complete response (CR), partial response (PR), stable disease (SD), and POD. A, Red indicates a positive association with the outcome (value representing Spearman’s rank correlation), whereas blue indicates a negative association. Only correlations with a significance threshold better than 0.1 and P values less than 0.1 are displayed. B, All correlation values for POD are displayed on the PAGA graph without restriction. C, All correlation values for CR are displayed on the PAGA graph without restriction. For B and C, the size of the circles is proportional to the P value (P value < 0.05 for larger blobs, 0.05 < P value < 0.1 for intermediate blobs, and P value > 0.1 for smaller blobs), and the color is proportional to the correlation.