Improved prediction of immune checkpoint blockade efficacy across multiple cancer types

Abstract

Only a fraction of patients with cancer respond to immune checkpoint blockade (ICB) treatment, but current decision-making procedures have limited accuracy. In this study, we developed a machine learning model to predict ICB response by integrating genomic, molecular, demographic and clinical data from a comprehensively curated cohort (MSK-IMPACT) with 1,479 patients treated with ICB across 16 different cancer types. In a retrospective analysis, the model achieved high sensitivity and specificity in predicting clinical response to immunotherapy and predicted both overall survival and progression-free survival in the test data across different cancer types. Our model significantly outperformed predictions based on tumor mutational burden, which was recently approved by the U.S. Food and Drug Administration for this purpose¹. Additionally, the model provides quantitative assessments of the model features that are most salient for the predictions. We anticipate that this approach will substantially improve clinical decision-making in immunotherapy and inform future interventions.

Fig. 1|. Overview of development of the model for integrated clinical-genetic prediction of ICB response.

a, Bar chart showing the number of patients in each of the 16 cancer types. We categorized response based on RECIST vl.1 (ref.) or best radiographic response. CR and PR were classified as R; SD and PD were classified as NR. Numbers in parentheses denote the number of patients in NR and R groups, respectively. b, General overview of the random forest model training and testing procedure. Sixteen cancer types were divided into training (80%) and testing (20%) subsets individually. A random forest model was trained on multiple genomic, molecular, demographic and clinical features on the training data using five-fold cross-validation to predict ICB response (NR and R). The resulting trained model with the best hyperparameters was evaluated using various performance metrics using the test set. c, Feature contribution of the 16 model features calculated in the training set (n = 1,184) to predict ICB response. The error bars denote standard deviation of feature contribution. d, ROC curves and the corresponding AUC values of RF16, RF11 and TMB alone in the training set across multiple cancer types. The numbers on the ROC curves denote the corresponding optimal cutpoints for RF16, which maximize the sensitivity and specificity of the response prediction. SCLC, small cell lung cancer.

Fig. 2|. Model performance across multiple cancer types in the test set.

a, ROC curves and corresponding AUC values of RF16, RF11 and TMB alone. b, Comparison of response probability distributions calculated by RF16 between NR and R groups. Two-sided P values were calculated using the Mann-Whitney U test. Center bar, median; box, interquartile range; whiskers, first and third quartiles ±1.5x interquartile range. c, Comparison of TMB between NR and R groups. Two-sided P values were calculated using the Mann-Whitney U test. Center bar, median; box, interquartile range; whiskers, first and third quartiles ±1.5x interquartile range. d-g, Confusion matrices showing predicted outcomes generated by RF16 and TMB, as indicated, in pan-cancer (d), in melanoma (e), in NSCLC (f) and in others (not melanoma/NSCLC) (g), respectively. To define high TMB tumors, we applied the threshold of ≥10 mut/Mb, which was approved by the FDA to predict ICB efficacy of solid tumors with pembrolizumab¹. h, Performance measurements of RF16, RF11 and TMB illustrated by sensitivity, specificity, accuracy, PPV and NPV.

Fig. 3|. Model predicts OS and PFS across multiple cancer types in the test set.

a, Comparison of C-index and 95% Cl for predicting OS among RF16, RF11 and TMB in the pan-cancer cohort (n = 295). b, Pan-cancer association between ICB response predicted by RF16 and OS. c, Comparison of C-index and 95% Cl for predicting OS among RF16, RF11 and TMB in melanoma (n = 37). d, Association between ICB response predicted by RF16 and OS in melanoma, e, Comparison of C-index and 95% Cl for predicting OS among RF16, RF11 and TMB in NSCLC (n=108). f, Association between ICB response predicted by RF16 and OS in NSCLC. g, Comparison of C-index and 95% Cl for predicting OS among RF16, RF11 and TMB in others (not melanoma/NSCLC) (n = 150). h, Association between ICB response predicted by RF16 and OS in others. i, Comparison of C-index and 95% Cl for predicting PFS among RF16, RF11 and TMB in the pan-cancer cohort (n = 295). j, Pan-cancer association between ICB response predicted by RF16 and PFS. k, Comparison of C-index and 95% Cl for predicting PFS among RF16, RF11 and TMB in melanoma (n = 37). l, Association between ICB response predicted by RF16 and PFS in melanoma. m, Comparison of C-index and 95% Cl for predicting PFS among RF16, RF11 and TMB in NSCLC (n = 108). n, Association between ICB response predicted by RF16 and PFS in NSCLC. o, Comparison of C-index and 95% Cl for predicting PFS among RF16, RF11 and TMB in others (n = 150). p, Association between ICB response predicted by RF16 and PFS in others. Two-sided P values for comparison of C-indices and survival times were computed using the paired Student’s t-test and log-rank test, respectively.