Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Jul;5(7):e404-e420.
doi: 10.1016/S2589-7500(23)00082-1. Epub 2023 May 31.

Predicting benefit from immune checkpoint inhibitors in patients with non-small-cell lung cancer by CT-based ensemble deep learning: a retrospective study

Affiliations

Predicting benefit from immune checkpoint inhibitors in patients with non-small-cell lung cancer by CT-based ensemble deep learning: a retrospective study

Maliazurina B Saad et al. Lancet Digit Health. 2023 Jul.

Erratum in

Abstract

Background: Only around 20-30% of patients with non-small-cell lung cancer (NCSLC) have durable benefit from immune-checkpoint inhibitors. Although tissue-based biomarkers (eg, PD-L1) are limited by suboptimal performance, tissue availability, and tumour heterogeneity, radiographic images might holistically capture the underlying cancer biology. We aimed to investigate the application of deep learning on chest CT scans to derive an imaging signature of response to immune checkpoint inhibitors and evaluate its added value in the clinical context.

Methods: In this retrospective modelling study, 976 patients with metastatic, EGFR/ALK negative NSCLC treated with immune checkpoint inhibitors at MD Anderson and Stanford were enrolled from Jan 1, 2014, to Feb 29, 2020. We built and tested an ensemble deep learning model on pretreatment CTs (Deep-CT) to predict overall survival and progression-free survival after treatment with immune checkpoint inhibitors. We also evaluated the added predictive value of the Deep-CT model in the context of existing clinicopathological and radiological metrics.

Findings: Our Deep-CT model demonstrated robust stratification of patient survival of the MD Anderson testing set, which was validated in the external Stanford set. The performance of the Deep-CT model remained significant on subgroup analyses stratified by PD-L1, histology, age, sex, and race. In univariate analysis, Deep-CT outperformed the conventional risk factors, including histology, smoking status, and PD-L1 expression, and remained an independent predictor after multivariate adjustment. Integrating the Deep-CT model with conventional risk factors demonstrated significantly improved prediction performance, with overall survival C-index increases from 0·70 (clinical model) to 0·75 (composite model) during testing. On the other hand, the deep learning risk scores correlated with some radiomics features, but radiomics alone could not reach the performance level of deep learning, indicating that the deep learning model effectively captured additional imaging patterns beyond known radiomics features.

Interpretation: This proof-of-concept study shows that automated profiling of radiographic scans through deep learning can provide orthogonal information independent of existing clinicopathological biomarkers, bringing the goal of precision immunotherapy for patients with NSCLC closer.

Funding: National Institutes of Health, Mark Foundation Damon Runyon Foundation Physician Scientist Award, MD Anderson Strategic Initiative Development Program, MD Anderson Lung Moon Shot Program, Andrea Mugnaini, and Edward L C Smith.

PubMed Disclaimer

Conflict of interest statement

Declaration of interests NIV receives consulting fees from Sanofi, Regeneron, Oncocyte, and Eli Lilly; and research funding from Mirati, outside the submitted work. SHL reports research funding from STCube Pharmaceuticals, Beyond Spring Pharmaceuticals, and Nektar Therapeutics; being on an advisory board for AstraZeneca and Creatv Microtech; and receiving consultation fees from XRAD Therapeutics, all outside the submitted work. PPL reports personal fees from Viewray and AstraZeneca; personal fees and non-financial support from Varian; and personal fees from Genentech, outside the submitted work. SG reports research support from AstraZeneca, BMS, and Millenium Pharmaceuticals, all outside the submitted work. JYC reports research funding from BMS-MDACC and consultation fees from Legion Healthcare Partners. MFG reports research support from Varian Medical Systems and RefleXion Medical. HAW reports research funding from ACEA Biosciences, Arrys Therapeutics, AstraZeneca/Medimmune, BMS, Clovis Oncology, Genentech/Roche, Merck, Novartis, SeaGen, Xcovery, and Helsinn; being on an advisory board for AstraZeneca, Blueprint, Mirati, Merck, and Genentech/Roche; and has leadership roles with the International Association for the Study of Lung Cancer, and ECOG-ACRIN. JWN reports honoraria from CME Matters, Clinical Care Options Continuing Medical Education (CME), Research to Practice CME, Medscape CME, Biomedical Learning Institute CME, MLI Peerview CME, Prime Oncology CME, Projects in Knowledge CME, Rockpointe CME, MJH Life Sciences CME, Medical Educator Consortium, and HMP Education; consulting or advisory roles for AstraZeneca, Genentech/Roche, Exelixis, Jounce Therapeutics, Takeda Pharmaceuticals, Eli Lilly, Calithera Biosciences, Amgen, Iovance Biotherapeutics, Blueprint Pharmaceuticals, Regeneron Pharmaceuticals, Natera, Sanofi/Regeneron, D2G Oncology, Surface Oncology, Turning Point Therapeutics, Mirati Therapeutics, Gilead Sciences, and AbbVie; and research funding from Genentech/Roche, Merck, Novartis, Boehringer Ingelheim, Exelixis, Nektar Therapeutics, Takeda Pharmaceuticals, Adaptimmune, GSK, Janssen, and AbbVie. H-SL reports research funding from Samyang Biopharmaceutical USA. VV reports consulting fees from BMS, Merck, Novartis, Amgen, Foundation Medicine, and AstraZeneca. YL reports research funding from Merck, MacroGenics, Tolero Pharmaceuticals, AstraZeneca, Vaccinex, Blueprint Medicines, Harpoon Therapeutics, Sun Pharma Advanced Research, Bristol Myers Squibb, Kyowa Pharmaceuticals, Tesaro, Bayer HealthCare, Mirati Therapeutics, and Daiichi Sankyo; has been on scientific advisory boards for AstraZeneca Pharmaceuticals, Janssen Pharmaceutical, Lilly Oncology, and Turning Point Therapeutics; has received consultation fees from AstraZeneca; and has received honoraria from Clarion Health Care. MP reports research funding from Novartis Institutes for Biomedical Research. XLe reports research funding from Eli Lilly, EMD Serono, Regeneron, and Boehringer Ingelheim; and consultant fees from EMD Serono (Merck KGaA), AstraZeneca, Spectrum Pharmaceutics, Novartis, Eli Lilly, Boehringer Ingelheim, Hengrui Therapeutics, Janssen, Blueprint Medicines, Sensei Biotherapeutics, and AbbVie, outside the submitted work. YYE discloses research support from AstraZeneca, Takeda, Eli Lilly, Xcovery, Tuning Point Therapeutics, Blueprint, Elevation Oncology, Spectrum, and Nuvalent; having advisory roles for AstraZeneca, Eli Lilly, Takeda, Specturm, Bristol Myers Squibb, and Turning Point Therapeutics; and accommodation expenses from Eli Lilly. MVN has been on scientific advisory boards for Mirati, Merck/MSD, and Genentech; and has received research funding from Mirati, Novartis, Checkmate, Alaunos/Ziopharm, AstraZeneca, Pfizer, and Genentech. FS reports consulting fees and advisory roles from Amgen, AstraZeneca Pharmaceuticals, Novartis, BeiGene, Tango Therapeutics, Calithera Biosciences, Navire Pharma, Medscape, Intellisphere, Guardant Health, and BergenBio; speaker fees from BMS, RV Mais Promoção e Eventos, Visiting Speakers Programme in Oncology at McGill University and the Université de Montréal, AIM Group International, and ESMO; fees for travel, food, and beverages from Tango Therapeutics, AstraZeneca Pharmaceuticals, Amgen, Guardant Health, and Dava Oncology; stock or stock options in BioNTech and Moderna; research grants (to institution) from Amgen, Mirati Therapeutics, Boehringer Ingelheim, Merck & Co, and Novartis; study chair funds (to institution) from Pfizer; and research grants (spouse, to institution) from Almmune. CMG reports fees for advisory committees from AstraZeneca, Bristol Myers Squibb, Jazz Pharmaceuticals, and Monte Rosa Therapeutics; research support from AstraZeneca; and speaker's fees from AstraZeneca and Beigene. TC reports speaker fees and honoraria from The Society for Immunotherapy of Cancer, Bristol Myers Squibb, Roche, Medscape, and PeerView; having an advisory role or receiving consulting fees from AstraZeneca, Bristol Myers Squibb, EMD Serono, Merck & Co, Genentech, and Arrowhead Pharmaceuticals; and institutional research funding from AstraZeneca, Bristol Myers Squibb, and EMD Serono. IIW reports grants and personal fees from Genentech/Roche, Bayer, Bristol Myers Squibb, AstraZeneca, Pfizer, HTG Molecular, Merck, Guardant Health, Novartis, and Amgen; personal fees from GSK, Flame, Sanofi, Daiichi Sankyo, Oncocyte, Janssen, MSD, and Platform Health; and grants from Adaptimmune, Adaptive, 4D, EMD Serono, Takeda, Karus, Iovance, Johnson & Johnson, and Akoya outside the submitted work. JDH is on the Scientific Advisory Board of Imagion Biosystems. DLG reports honoraria for scientific advisory boards from AstraZeneca, Sanofi, Alethia Biotherapeutics, Menarini, Eli Lilly, 4D Pharma, and Onconova; and research support from Janssen, Takeda, Astellas, Ribon Therapeutics, NGM Biopharmaceuticals, Boehringer Ingelheim, Mirati Therapeutics, and AstraZeneca. JVH reports being on scientific advisory boards for AstraZeneca, Boehringer Ingelheim, Genentech, GlaxoSmithKline, Eli Lilly, Novartis, Spectrum, EMD Serono, Sanofi, Takeda, Mirati Therapeutics, BMS, and Janssen Global Services; receiving research support from AstraZeneca, Takeda, Boehringer Ingelheim, and Spectrum; and receiving licensing fees from Spectrum. JZ reports research funding from Merck, Johnson & Johnson, and Novartis; and consultant fees from BMS, Johnson & Johnson, AstraZeneca, Geneplus, OrigMed, Novartis, and Innovent, outside the submitted work. CCW reports research support from Medical Imaging and Data Resource Center from NIBIB/University of Chicago and royalties from Elsevier. All other authors declare no competing interests.

Figures

Figure 1:
Figure 1:. Overall study design
(A) Schematic plot for developing the CT deep learning model and evaluating it in a clinical context. (B) The details of different model development and integration. Parts of figure created with BioRender (biorender.com). Conv + BN + ReLu=Convolution plus batch normalisation plus rectified linear unit. ECOG=Eastern Cooperative Oncology group. NSCLC=non-small-cell lung cancer.
Figure 2:
Figure 2:. Prognostic performance of the ensemble deep learning CT model (Deep-CT)
Kaplan-Meier survival curves for the MD Anderson cohort for (A) overall survival for training, (B) overall survival for validation, (C) overall survival for testing, (D) progression-free survival for training, (E) progression-free survival for validation, and (F) progression-free survival for testing. (G) Kaplan-Meier survival curve for external validation on the Stanford set. Joint stratification by Deep-CT and PD-L1 expression levels in patients treated with (H) immune checkpoint inhibitor monotherapy and (I) combination immune checkpoint inhibitor plus chemotherapy. HR=hazard ratio.
Figure 3:
Figure 3:. Radiological interpretation of the CT deep learning models
(A) Venn diagrams representing the number of features that showed significant correlation with the four individual subnetwork models and their ensemble model, with positive correlation and negative correlation. The red colour denotes the features agreed to be significant by more than one model. (B) Visualisation of different prognostic information captured by ensemble Deep-CT on selected four patients. The charts show radiomic feature score distribution. The first and second columns of scans show the first pair of patients (high-risk vs low-risk predicted by Deep-CT), and the last two columns of scans show another set of patients (high-risk vs low-risk). Each row shows how a feature score corresponds to radiological characteristic as seen in CT images. Each pair of patients were reported with roughly similar size of primary tumour volume and overall tumour burden, but with significant difference in overall survival and progression-free survival. The Deep-CT model captured non-conventional prognostic information including textural complexity and heterogeneity in primary tumour and background lung parenchyma beyond disease burden. ICI=immune checkpoint inhibitor.
Figure 4:
Figure 4:. Evaluation of Deep-CT in clinical context
Forest plots summarise the univariate and multivariate analyses for the CT-deep learning model. (A) Overall survival. (B) Progression-free survival. Blue data points are from univariate analysis. Red datapoints are from multivariate analysis. Multivariate analysis of progression-free survival used smaller cohort sizes in the discovery cohort (n=383) and the testing cohort (n=161) due to missing values. The exact p values are shown in the appendix (pp 25–26). ECOG PS=Eastern Cooperative Oncology Group Performance Status. ICI=immune checkpoint inhibitor. *p<0·001. †For overall survival, PD-L1 expression used smaller cohort sizes in the discovery cohort (n=463) and testing cohort (n=186) due to missing values. ‡p<0·05. §p<0·01. ¶For overall survival, ECOG PS used smaller cohort sizes in the discovery cohort (n=512) and testing cohort (n=221) due to missing values.
Figure 5:
Figure 5:. Added prognostic value of the Deep-CT model on the clinical benchmark model
The Kaplan-Meier survival curves jointly stratified by the Deep-CT model and the benchmark model for (A) overall survival training, (B) overall survival validation, (C) overall survival testing, (D) progression-free survival training, (E) progression-free survival validation, and (F) progression-free survival testing. The composite model integrating the Deep-CT model and the benchmark model for (G) overall survival training, (H) overall survival validation, (I) overall survival testing, (J) progression-free survival training, (K) progression-free survival validation, and (L) progression-free survival testing.
Figure 5:
Figure 5:. Added prognostic value of the Deep-CT model on the clinical benchmark model
The Kaplan-Meier survival curves jointly stratified by the Deep-CT model and the benchmark model for (A) overall survival training, (B) overall survival validation, (C) overall survival testing, (D) progression-free survival training, (E) progression-free survival validation, and (F) progression-free survival testing. The composite model integrating the Deep-CT model and the benchmark model for (G) overall survival training, (H) overall survival validation, (I) overall survival testing, (J) progression-free survival training, (K) progression-free survival validation, and (L) progression-free survival testing.
Figure 6:
Figure 6:. C-index comparison of six different prediction models for overall survival and progression-free survival
C-index comparison of six different prediction models (PD-L1, overall disease volume, radiomics model, Deep-CT, benchmark model, and the composite model that integrates Deep-CT and benchmark models) for (A) overall survival and (B) progression-free survival. The comparison was made on a smaller cohort size that contains the overlapped population with PD-L1 expression.

Comment in

References

    1. Siegel RL, Miller KD, Fuchs HE, Jemal A. Cancer statistics, 2022. CA Cancer J Clin 2022; 72: 7–33. - PubMed
    1. Ribas A, Wolchok JD. Cancer immunotherapy using checkpoint blockade. Science 2018; 359: 1350–55. - PMC - PubMed
    1. Camidge DR, Doebele RC, Kerr KM. Comparing and contrasting predictive biomarkers for immunotherapy and targeted therapy of NSCLC. Nat Rev Clin Oncol 2019; 16: 341–55. - PubMed
    1. Doroshow DB, Bhalla S, Beasley MB, et al. PD-L1 as a biomarker of response to immune-checkpoint inhibitors. Nat Rev Clin Oncol 2021; 18: 345–62. - PubMed
    1. Hellmann MD, Ciuleanu TE, Pluzanski A, et al. Nivolumab plus ipilimumab in lung cancer with a high tumor mutational burden. N Engl J Med 2018; 378: 2093–104. - PMC - PubMed

Publication types

MeSH terms