Multi-Size Deep Learning Based Preoperative Computed Tomography Signature for Prognosis Prediction of Colorectal Cancer

Abstract

Background: Preoperative and postoperative evaluation of colorectal cancer (CRC) patients is crucial for subsequent treatment guidance. Our study aims to provide a timely and rapid assessment of the prognosis of CRC patients with deep learning according to non-invasive preoperative computed tomography (CT) and explore the underlying biological explanations. Methods: A total of 808 CRC patients with preoperative CT (development cohort: n = 426, validation cohort: n = 382) were enrolled in our study. We proposed a novel end-to-end Multi-Size Convolutional Neural Network (MSCNN) to predict the risk of CRC recurrence with CT images (CT signature). The prognostic performance of CT signature was evaluated by Kaplan-Meier curve. An integrated nomogram was constructed to improve the clinical utility of CT signature by combining with other clinicopathologic factors. Further visualization and correlation analysis for CT deep features with paired gene expression profiles were performed to reveal the molecular characteristics of CRC tumors learned by MSCNN in radiographic imaging. Results: The Kaplan-Meier analysis showed that CT signature was a significant prognostic factor for CRC disease-free survival (DFS) prediction [development cohort: hazard ratio (HR): 50.7, 95% CI: 28.4-90.6, p < 0.001; validation cohort: HR: 2.04, 95% CI: 1.44-2.89, p < 0.001]. Multivariable analysis confirmed the independence prognostic value of CT signature (development cohort: HR: 30.7, 95% CI: 19.8-69.3, p < 0.001; validation cohort: HR: 1.83, 95% CI: 1.19-2.83, p = 0.006). Dimension reduction and visualization of CT deep features demonstrated a high correlation with the prognosis of CRC patients. Functional pathway analysis further indicated that CRC patients with high CT signature presented down-regulation of several immunology pathways. Correlation analysis found that CT deep features were mainly associated with activation of metabolic and proliferative pathways. Conclusions: Our deep learning based preoperative CT signature can effectively predict prognosis of CRC patients. Integration analysis of multi-omic data revealed that some molecular characteristics of CRC tumor can be captured by deep learning in CT images.

Keywords: colorectal cancer; deep learning; nomogram; pathway analysis; prognosis.

FIGURE 1

Workflow of MSCNN. (A) Multi-Size based data enhancement of CT images before fed into MSCNN. (B) Data preprocessing of CT images with ROIs. (C) Network structure of MSCNN Multi-Size which includes a CNN to combine Multi-Size CT data, a ResNet34 network to extract image features of tumors from CT images and a last classification network.

FIGURE 2

Prognostic performance of MSCNN. The distribution of CT signature of MSCNN and its corresponding recurrence status in the development cohort (A) and validation cohort (C). Kaplan-Meier curves showed a significant survival difference between the high and low risk groups in the development cohort (B) and validation cohort (D). Prognostic analysis of CRC patients in stage II and III subgroups (E–H). Univariable and multivariable analysis of clinical factors in the development cohort (I) and validation cohort (J).

FIGURE 3

The developed nomogram incorporated CT signature with T & N stage (A). Coordinates length for each prognostic factor was determined by the coefficients of the cox regression model. For each patient, the total score was calculated with all variable scores. The probability of DFS was derived from the mapping relationship between the evaluation results and total score on specified patient survival time. (B,C) Calibration curves of nomogram for 5 years DFS in the development and validation cohort. (D,E) Decision curve analysis for nomogram established in the development and validation cohort.

FIGURE 4

Dimension reduction for visualization and correlation analysis of deep CT features. Principle component analysis (PCA) on the 512 features of the ResNet34 network (A,C) and 64 features (CT feature) of hidden notes of the FC network (B,D). Correlation heatmap between 64 deep CT features and prognostic difference group (E).

FIGURE 5

Global gene set pathway analysis. (A) Gene Ontology pathway enrichment analysis between CT signatures and RNA-Seq expression. (B) GSEA showed several Immune related pathways were downregulated in high CT signature patients. (C,D) Correlation between 64 deep CT features and their enrichment hallmark pathways.