Deep Transfer Learning and Radiomics Feature Prediction of Survival of Patients with High-Grade Gliomas

Abstract

Background and purpose: Patient survival in high-grade glioma remains poor, despite the recent developments in cancer treatment. As new chemo-, targeted molecular, and immune therapies emerge and show promising results in clinical trials, image-based methods for early prediction of treatment response are needed. Deep learning models that incorporate radiomics features promise to extract information from brain MR imaging that correlates with response and prognosis. We report initial production of a combined deep learning and radiomics model to predict overall survival in a clinically heterogeneous cohort of patients with high-grade gliomas.

Materials and methods: Fifty patients with high-grade gliomas from our hospital and 128 patients with high-grade glioma from The Cancer Genome Atlas were included. For each patient, we calculated 348 hand-crafted radiomics features and 8192 deep features generated by a pretrained convolutional neural network. We then applied feature selection and Elastic Net-Cox modeling to differentiate patients into long- and short-term survivors.

Results: In the 50 patients with high-grade gliomas from our institution, the combined feature analysis framework classified the patients into long- and short-term survivor groups with a log-rank test P value < .001. In the 128 patients from The Cancer Genome Atlas, the framework classified patients into long- and short-term survivors with a log-rank test P value of .014. For the mixed cohort of 50 patients from our institution and 58 patients from The Cancer Genome Atlas, it yielded a log-rank test P value of .035.

Conclusions: A deep learning model combining deep and radiomics features can dichotomize patients with high-grade gliomas into long- and short-term survivors.

Fig 1.

A, Flowchart shows our survival prediction system. B, The framework of VGG-19 model for deep-feature extraction.

Fig 2.

Example of contrast-enhanced T1-weighted MR images of longer term survivors (A and B) with an overall survival of 1405 days and shorter term survivors (C and D) with an overall survival of 447 days. A and C, Contrast-enhanced T1-weighted MR images with tumor contours in red. B and D, Tumor patches segmented from A and C, respectively.

Fig 3.

Kaplan-Meier curve of predicted longer term and shorter term survival in a dataset (A) with 50 patients with HGG from Brigham and Women’s Hospital, a dataset (B) with 128 patients with HGG from the TCGA, and a dataset (C) with 108 patients with HGG from Brigham and Women’s Hospital and the TCGA.