Machine learning for prediction of immunotherapeutic outcome in non-small-cell lung cancer based on circulating cytokine signatures

Abstract

Background: Immune checkpoint inhibitor (ICI) therapy has substantially improved the overall survival (OS) in patients with non-small-cell lung cancer (NSCLC); however, its response rate is still modest. In this study, we developed a machine learning-based platform, namely the Cytokine-based ICI Response Index (CIRI), to predict the ICI response of patients with NSCLC based on the peripheral blood cytokine profiles.

Methods: We enrolled 123 and 99 patients with NSCLC who received anti-PD-1/PD-L1 monotherapy or combined chemotherapy in the training and validation cohorts, respectively. The plasma concentrations of 93 cytokines were examined in the peripheral blood obtained from patients at baseline (pre) and 6 weeks after treatment (early during treatment: edt). Ensemble learning random survival forest classifiers were developed to select feature cytokines and predict the OS of patients undergoing ICI therapy.

Results: Fourteen and 19 cytokines at baseline and on treatment, respectively, were selected to generate CIRI models (namely preCIRI14 and edtCIRI19), both of which successfully identified patients with worse OS in two completely independent cohorts. At the population level, the prediction accuracies of preCIRI14 and edtCIRI19, as indicated by the concordance indices (C-indices), were 0.700 and 0.751 in the validation cohort, respectively. At the individual level, patients with higher CIRI scores demonstrated worse OS [hazard ratio (HR): 0.274 and 0.163, and p<0.0001 and p=0.0044 in preCIRI14 and edtCIRI19, respectively]. By including other circulating and clinical features, improved prediction efficacy was observed in advanced models (preCIRI21 and edtCIRI27). The C-indices in the validation cohort were 0.764 and 0.757, respectively, whereas the HRs of preCIRI21 and edtCIRI27 were 0.141 (p<0.0001) and 0.158 (p=0.038), respectively.

Conclusions: The CIRI model is highly accurate and reproducible in determining the patients with NSCLC who would benefit from anti-PD-1/PD-L1 therapy with prolonged OS and may aid in clinical decision-making before and/or at the early stage of treatment.

Keywords: Biomarkers, Tumor; Biostatistics; Cytokines; Immune Checkpoint Inhibitors; Non-Small Cell Lung Cancer.

Figure 1

Overview of a circulating cytokine-based machine learning approach for prediction of immunotherapeutic outcome in patients with non-small-cell lung cancer (NSCLC). AUC, area under the curve; CIRI, Cytokine-based ICI Response Index; OS, overall survival; ROC, receiver operating characteristic.

Figure 2

Prediction performance of preCIRI14 model using baseline circulating cytokine signature. (A) Feature cytokines selected using random survival forest (RSF) minimal depth filter: the importance score was defined as 1/(minimal depth); (B) Concordance (C)-indices of preCIRI14 model for the training and validation sets; time-dependent area under the curve AUC of receiver operating characteristic curves (ROCs) at each time point for the (C) training and (E) validation sets; overall survival time of ‘high risk’ and ‘low risk’ groups predicted by preCIRI14 for (D) training and (F) validation sets using cut-off value optimized by training set (see online supplemental figure S1). The CIRI scores indicate risk of event occurrence, whereas ‘high risk’ indicates patients with high CIRI scores and ‘low risk’ indicates patients with low CIRI scores.

Figure 3

Prediction performance of edtCIRI19 model using circulating cytokine signature early during treatment. (A) Feature cytokines selected using RSF minimal depth filter: the importance score was defined as 1/(minimal depth); (B) C-indices of edtCIRI19 model for training and validation sets; time-dependent AUC of ROC at each time points for the (C) training and (E) validation sets; overall survival time of ‘high risk’ (high CIRI) and ‘low risk’ (low CIRI) groups predicted by edtCIRI19 for the (D) training and (F) validation sets using cut-off values optimized by the training set (see online supplemental figure S3). AUC, area under the curve; CIRI, Cytokine-based ICI Response Index; ROC, receiver operating characteristic; RSF, random survival forest.

Figure 4

Prediction performance of the preCIRI21 model. Fourteen feature cytokines of preCIRI14 model and 7 clinical and circulating factors (age, sex, stage, BMI, blood albumin of pre, NLR of pre, and tumor PD-L1 expression) were used in preCIRI21. (A) C-indices of preCIRI21 model for the training and validation sets; time-dependent AUC of ROC at each time points for the (B) training and (D) validation sets; overall survival time of ‘high risk’ (high CIRI) and ‘low risk’ (low CIRI) groups predicted by preCIRI21 for the (C) training and (E) validation sets using optimized cut-off values (see online supplemental figure S6). AUC, area under the curve; BMI, body mass index; CIRI, Cytokine-based ICI Response Index; NLR, neutrophil-to-lymphocyte ratio; ROC, receiver operating characteristic.

Figure 5

Prediction performance of the edtCIRI27 model. Nineteen feature cytokines of edtCIRI21 model and 8 clinical and circulating factors (age, sex, stage, BMI, blood albumin of edt, NLR of edt, tumor PD-L1 expression, and treatment option) were used in edtCIRI27. (A) C-indices of edtCIRI27 model for the training and validation sets; time-dependent AUC of ROC at each time points for the (B) training and (D) validation sets; overall survival time of ‘high risk’ (high CIRI) and ‘low risk’ (low CIRI) groups predicted by preCIRI21 for the (C) training and (E) validation sets using optimized cut-off values (see online supplemental figure S7). AUC, area under the curve; BMI, body mass index; CIRI, Cytokine-based ICI Response Index; edt, early during treatment; NLR, neutrophil-to-lymphocyte ratio; ROC, receiver operating characteristic.

Figure 6

Association between feature cytokines and predicted Cytokine-based ICI Response Index (CIRI) scores. Spearman’s rank correlation coefficient between the feature cytokine levels and CIRI scores predicted by the (A) preCIRI14 and (B) edtCIRI19 models of cohort 1 and cohort 2. Significance levels: *p≤0.05, **p≤0.01, ***p≤0.001. ICI, immune checkpoint inhibitor.