Two-stage Cox-nnet: biologically interpretable neural-network model for prognosis prediction and its application in liver cancer survival using histopathology and transcriptomic data

Abstract

Pathological images are easily accessible data with the potential of prognostic biomarkers. Moreover, integration of heterogeneous data types from multi-modality, such as pathological image and gene expression data, is invaluable to help predicting cancer patient survival. However, the analytical challenges are significant. Here, we take the hepatocellular carcinoma (HCC) pathological image features extracted by CellProfiler, and apply them as the input for Cox-nnet, a neural network-based prognosis prediction model. We compare this model with the conventional Cox proportional hazards (Cox-PH) model, CoxBoost, Random Survival Forests and DeepSurv, using C-index and log-rank P-values. The results show that Cox-nnet is significantly more accurate than Cox-PH and Random Survival Forests models and comparable with CoxBoost and DeepSurv models, on pathological image features. Further, to integrate pathological image and gene expression data of the same patients, we innovatively construct a two-stage Cox-nnet model, and compare it with another complex neural-network model called PAGE-Net. The two-stage Cox-nnet complex model combining histopathology image and transcriptomic RNA-seq data achieves much better prognosis prediction, with a median C-index of 0.75 and log-rank P-value of 6e-7 in the testing datasets, compared to PAGE-Net (median C-index of 0.68 and log-rank P-value of 0.03). Imaging features present additional predictive information to gene expression features, as the combined model is more accurate than the model with gene expression alone (median C-index 0.70). Pathological image features are correlated with gene expression, as genes correlated to top imaging features present known associations with HCC patient survival and morphogenesis of liver tissue. This work proposes two-stage Cox-nnet, a new class of biologically relevant and interpretable models, to integrate multiple types of heterogenous data for survival prediction.

Figure 1.

The architectures of Cox-nnet model: the sketch of Cox-nnet model for prognosis prediction, based on a single data type.

Figure 2.

Comparison of prognosis prediction among different models using pathology imaging data. (A) C-index results on training and testing datasets. (B–F) Kaplan–Meier survival curves on testing datasets using different methods. (B) Cox-nnet (C) CoxBoost (D) DeepSurv (E) Cox-PH (F) Random Survival Forests (RSF).

Figure 3.

The architectures of two-stage Cox-nnet. The first stage builds individual Cox-nnet models for each data type. The second stage combines the hidden nodes from the first stage Cox-nnet models as the input, and builds a new Cox-nnet model.

Figure 4.

Comparison of Cox-nnet prognosis prediction using pathology imaging, gene expression and the combination of the two data types. (A) C-index results on training and testing datasets. (B–D) Kaplan–Meier survival curves on testing datasets. (B) Cox-nnet model on imaging data only. (C) Cox-nnet model on gene expression data only. (D) two-stage Cox-nnet model combining imaging and gene expression data.

Figure 5.

Comparison of two-stage Cox-nnet and PAGE-Net, based on combined pathological images and gene expression. (A) C-index of the two methods on training (red) and testing (blue) datasets, on 20 repetitions. (B–E) Kaplan–Meier survival curves resulting from the Cox-nnet (B and D) and PAGE-Net model (C and E) using training and testing datasets, respectively.

Figure 6.

Relationship between top imaging and gene features. Rectangle nodes are image features and circle nodes are gene features. Node sizes are proportional to importance scores from Cox-nnet. Two gene nodes are connected only if their correlation is > 0.5; an image node and a gene node are connected only if their correlation is > 0.1 (P-value < 0.05). Green nodes represent features with positive coefficients (hazard ratio) in univariate Cox-PH regression, indicating worse prognosis. Blue nodes represent features with negative coefficients (hazard ratio) in univariate Cox-PH regression, indicating protection against bad prognosis.