HIBRID: histology-based risk-stratification with deep learning and ctDNA in colorectal cancer

Abstract

Although surgical resection is the standard therapy for stage II/III colorectal cancer, recurrence rates exceed 30%. Circulating tumor DNA (ctNDA) detects molecular residual disease (MRD), but lacks spatial and tumor microenvironment information. Here, we develop a deep learning (DL) model to predict disease-free survival from hematoxylin & eosin stained whole slide images in stage II-IV colorectal cancer. The model is trained on the DACHS cohort (n = 1766) and validated on the GALAXY cohort (n = 1404). In GALAXY, the DL model categorizes 304 patients as DL high-risk and 1100 as low-risk (HR 2.31; p < 0.005). Combining DL scores with MRD status improves prognostic stratification in both MRD-positive (HR 1.58; p < 0.005) and MRD-negative groups (HR 2.1; p < 0.005). Notably, MRD-negative patients predicted as DL high-risk benefit from adjuvant chemotherapy (HR 0.49; p = 0.01) vs. DL low-risk (HR = 0.92; p = 0.64). Combining ctDNA with DL-based histology analysis significantly improves risk stratification, with the potential to improve follow-up and personalized adjuvant therapy decisions.

Fig. 1. Study Design and DL Model Architecture.

A DACHS cohort and B GALAXY cohort overview including patient characteristics and WSI preprocessing pipeline using UNI, a pretrained vision encoder for feature extraction. C Overview Experimental Setup: Clinical data is fed into DL Model with WSIs for training process and then externally deployed onto the GALAXY cohort to obtain the DL-Score, which are then binarized into DL high-risk and DL low-risk categories. D Architecture of the Transformer-based Multiple Instance Learning (MIL) pipeline. WSIs are divided into patches and preprocessed to feature vectors with a dimension of n-tiles x1024 using the UNI foundation model. Patch feature vectors are then projected to a 512-dimensional vector using a fully connected layer with ReLU activation, with a learnable class token (CLS) added. A two-layer transformer refines the CLS token via self-attention and feedforward networks. The final CLS token, encoding WSI-level information, is processed by an MLP to generate the patient-level risk score. This Figure was partly generated using Flaticon. DACHS=Darmkrebs: Chancen der Verhütung durch Screening Study, WSI=whole-slide image, DFS=disease-free survival, DL=Deep Learning, MRD=molecular residual disease, CLS=class learnable token, MLP=multilayer perceptron.

Fig. 2. DL and ctDNA-status stratify patients by recurrence risk.

A Kaplan-Meier curves for DFS stratified by DL high-risk and DL low-risk patients. B Kaplan-Meier curves for DFS stratified by MRD-positive and MRD-negative patients. C Forest plot showing multivariate cox regression analysis including the covariates gender (p = 0.70), age (p = 0.06), DL risk score (p = 0.06), pathological nodal stage (pN), pathological tumor stage (pT; p = 0.20), metastasis stage (pM), adjuvant chemotherapy treatment (ACT), microsatellite instability status (MSI; p = 0.01), and MRD-status, and their association with DFS. D Kaplan-Meier curves for DFS stratified into three distinct risk categories: Double High Risk (MRD-positive and DL high-risk), Either High Risk (either MRD-positive/DL low-risk or MRD-negative/DL high-risk), and Double Low Risk (MRD-negative/DL low-risk). HR and 95% CI were calculated by the Cox proportional hazard model. P value was calculated using the two-sided log-rank test (*p < 0.05, **p < 0.005). P values < 0.005 are not listed individually. Each Kaplan–Meier analysis was performed once using the full cohort and reflects the entire dataset. No subsampling or repeated trials were applied. Plots were generated using the lifelines package in Python 3.11.5. Source data is provided as a Source Data file. DACHS=Darmkrebs: Chancen der Verhütung durch Screening Study, WSI=whole-slide image, DFS=disease-free survival, DL=Deep Learning, MRD=molecular residual disease, HR=Hazard ratio, CI=Confidence interval.

Fig. 3. DL stratifies recurrence risk within MRD subgroups.

Kaplan-Meier curves showing DFS stratification by DL high-risk and DL low-risk groups for A MRD-positive and B MRD-negative groups, followed by Kaplan-Meier curves showing DFS stratified by with or without ACT treatment in C MRD-positive and DL low-risk, D MRD-negative and DL low-risk, E MRD-positive and DL high-risk and F MRD-negative and DL high-risk subgroups. HR and 95% CI were calculated by the Cox proportional hazard model. P value was calculated using the two-sided log-rank test. Plots were generated using the lifelines package in Python 3.11.5 Source data is provided as a Source Data file. DFS=disease-free survival, DL=Deep Learning, ACT=adjuvant chemotherapy, MRD=molecular residual disease, HR=Hazard ratio, CI=Confidence interval.

Fig. 4. DL can identify morphological features linked to prognosis.

A Highly predictive tiles for patients below the DL risk-threshold and above the DL risk-threshold exemplarily with DL score reported. B Whole slide patient heatmaps showing the DL prediction score, red indicating high-risk, and blue indicating low-risk. Visualizations are based on a single analysis run and represent typical patterns observed in the dataset. DL=Deep Learning, MRD=molecular residual disease.