Deep Learning Radiopathomics Models Based on Contrast-enhanced MRI and Pathologic Imaging for Predicting Vessels Encapsulating Tumor Clusters and Prognosis in Hepatocellular Carcinoma

Abstract

Purpose To develop deep learning (DL) radiopathomics models based on contrast-enhanced MRI and pathologic imaging to predict vessels encapsulating tumor clusters (VETC) and survival in hepatocellular carcinoma (HCC). Materials and Methods In this retrospective, multicenter study, 578 patients with HCC (mean age [±SD], 59 years ± 10; 442 male, 136 female) were divided into the training (n = 317), internal (n = 137), and external (n = 124) test sets. DL radiomics and pathomics models were developed to predict VETC using gadoxetic acid-enhanced MR and pathologic images. Deep radiomics score (DRS) and handcrafted and deep pathomics scores were compared between the group with VETC pattern in HCC (VETC+) and group without VETC pattern in HCC (VETC-). Multivariable Cox regression analyses were performed to identify independent prognostic factors, and the radiopathomics nomogram models were developed for early recurrence and progression-free survival (PFS). The prognostic power was evaluated using the concordance index (C index) and time-dependent receiver operating characteristic (ROC) curves. Results In the external test set, the Swin Transformer showed good performance for predicting VETC in both DL radiomics (area under the ROC curve [AUC], 0.77-0.79) and pathomics (AUC, 0.79) models. Patients with VETC+ HCC had significantly higher DRS and handcrafted and deep pathomics scores compared with patients with VETC- HCC in all datasets (all P < .001). The radiopathomics nomogram model incorporating DRS in the arterial phase and the handcrafted and deep pathomics scores achieved C indexes of 0.69, 0.60, and 0.67 for early recurrence and time-dependent AUCs of 0.83 (95% CI: 0.76, 0.91), 0.81 (95% CI: 0.68, 0.94), and 0.78 (95% CI: 0.67, 0.88) for 3-year PFS in the training, internal, and external test sets, respectively. Early recurrence and PFS rates statistically significantly differed between the high- and low-risk patients stratified by the radiopathomics nomogram model (all P < .05). Conclusion DL radiopathomics models effectively helped to predict VETC in HCC and assess the risk for early recurrence and PFS. Keywords: Hepatocellular Carcinoma, Deep Learning, MRI, Radiopathomics, Survival Supplemental material is available for this article. © RSNA, 2025.

Keywords: Deep Learning; Hepatocellular Carcinoma; MRI; Radiopathomics; Survival.

Figure 1:

Patient selection and study design. (A) Flowchart of the patient selection process from three medical centers. (B) Workflow for deep learning radiopathomics model to predict vessels encapsulating tumor clusters and progression-free survival in hepatocellular carcinoma. Gd-EOB-DTPA = gadolinium-ethoxybenzyl-diethylenetriamine pentaacetic acid, HCC = hepatocellular carcinoma, VETC = vessels encapsulating tumor clusters.

Figure 2:

Handcrafted pathomics feature extraction and selection. (A) The most representative hematoxylin-eosin (HE)–stained image (original magnification, ×100) of the hepatocellular carcinoma (HCC) sample was input into CellProfiler software. (B, C) The HE-stained image was split into hematoxylin-stained and eosin-stained grayscale images. (D–G) Unmixed images were automatically segmented to identify the nuclei, cell, and cytoplasm. (H) LASSO (least absolute shrinkage and selection operator) coefficients profiles (y-axis) of the handcrafted pathomics features. The lower x-axis indicates the λ value. (I) Fifteen handcrafted pathomics features were selected into the LASSO model using 10-fold cross-validation when λ was the minimum loss.

Figure 3:

Deep learning radiopathomics models visualization. (A–D) Gradient-weighted class activation mapping (Grad-CAM) heatmaps of ResNet50, DenseNet121, Vision Transformer, and Swin Transformer models are shown based on the axial arterial-phase images, portal venous phase images, hepatobiliary phase images, and hematoxylin-eosin–stained images (original magnification, ×100) in a 71-year-old female patient with hepatocellular carcinoma. Red designates the activation region associated with vessels completely encapsulating tumor clusters.

Figure 4:

Performance of deep learning pathomics models and box plots of the pathomics scores. (A–C) Receiver operating characteristic (ROC) curves of the deep learning pathomics models for predicting vessels encapsulating tumor clusters (VETC) in the training, internal, and external test sets. (D–F) Box plots of the handcrafted pathomics scores between VETC+ and VETC− hepatocellular carcinoma (HCC) in the training, internal, and external test sets. (G–I) Box plots of deep pathomics scores between VETC+ and VETC− HCC in training, internal, and external test sets. The yellow and green box plots represent VETC+ and VETC− HCC, respectively. The box represents IQR and the median is indicated by the line inside the box. Whiskers extend to the smallest and largest values within 1.5 × IQR from the lower and upper hinges, respectively. Jittered points represent individual observations. Statistical significance was assessed using the Mann–Whitney U test.

Figure 5:

Construction and evaluation of radiopathomics nomograms. (A–C) Forest plot, radiopathomics nomogram, and decision curve for early recurrence. (D–F) Forest plot, radiopathomics nomogram, and decision curve for progression-free survival. The variables in the forest plot were selected based on the stepwise regression with Akaike information criterion. AFP = α-fetoprotein, AST = aspartate aminotransferase, DRS_AP = deep radiomics score in arterial phase.

Figure 6:

Prognostic value of the radiopathomics nomogram. (A–C) Early recurrence-free survival and (D–F) progression-free survival curves for the patients with high- and low-risk scores stratified by radiopathomics nomograms in the training, internal, and external test sets. (G–I) Time-dependent receiver operating characteristic (ROC) curves of the nomogram for progression-free survival in the training, internal, and external test sets. AUC = area under the ROC curve.