A multimodal AI model for precision prognosis in clear cell renal cell carcinoma: A multicenter study

Abstract

Patients with clear cell renal cell carcinoma (ccRCC) face a high risk of recurrence after surgery, but existing clinical tools based on clinicopathological factors or costly molecular profiling often lack precision and clinical feasibility. We developed the multimodal predictive recurrence score (MPRS), a multimodal prognostic model using clinical features, CT images, and histopathological whole-slide images (WSIs) from 1648 patients across six centers and the TCGA database. MPRS outperformed unimodal models and clinical tools (Leibovich and UISS scores, KEYNOTE-564 risk classification), achieving C-index values of 0.886 and 0.838 in the internal and external validation cohorts, respectively. Importantly, MPRS correctly reclassified 83.3% (50/60) of KEYNOTE-564-defined low-risk recurrence patients as high-risk, avoiding inadequate adjuvant therapy, while reclassifying 57.7% (15/26) of KEYNOTE-564-defined intermediate/high-risk non-recurrence patients as low-risk, preventing excessive adjuvant therapy. By leveraging routinely available data, MPRS provides a cost-effective and accurate approach for recurrence risk stratification, optimizing personalized ccRCC management and therapeutic decision-making.

Fig. 1. Study workflow.

Multimodal data, including clinical features, preoperative contrast-enhanced CT images, and histopathological whole-slide images (WSIs) of tumor sections, were acquired through conventional diagnostic procedures. The multimodal data were analyzed by means of numerous distinct approaches, including Cox regression, PyRadiomics features, ResNet, and HistomicsTK features. The clinical, radiological and histopathological models were fused to construct a multimodal predictive recurrence score (MPRS) that was used for the postoperative risk stratification of ccRCC patients, which was validated in multicenter cohorts. ccRCC=clear cell renal cell carcinoma. CT = computed tomography. WSI=whole slide image.

Fig. 2. Model performance.

a–c Bar plots depict the C-Index values for the MPRS, HPRS, RPRS, CPRS, Leibovich score, UISS score, and KEYNOTE-564 risk classification in patients with ccRCC in the training cohort, internal validation cohort, and external validation cohort. The C-index and its associated 95% CI were calculated using the bootstrap method with 1000 resamples. In each plot, the height of the bar represents the C-index value, and the error bars represent the 95% CI. d–f Calibration curves for the MPRS on the basis of the concordance between the predicted and observed DFS rates for ccRCC patients at 3 and 5 years. The diagonal reference line represents perfect calibration; deviations above this line indicate underestimation of risk, while deviations below indicate overestimation. The blue and red curves with error bars (95% CI) represent bootstrapped 3-year and 5-year observed (Y-axis) versus predicted probabilities (X-axis), respectively. g–i Kaplan–Meier analysis of the MPRS for DFS in patients with ccRCC, divided into low-risk and high-risk groups according to the risk score. p values were calculated with the log-rank test. ROC = receiver operating characteristic. AUC = area under the curve. MPRS = multimodal predictive recurrence score. HPRS = histopathological predictive recurrence score. RPRS = radiological predictive recurrence score. CPRS = clinical predictive recurrence score. KEYNOTE-564 = KEYNOTE-564 group. ccRCC = clear cell renal cell carcinoma. DFS = disease-free survival.

Fig. 3. Risk stratification analysis for DFS of the MPRS on the basis of the KEYNOTE-564 risk classification subgroups.

a Sankey diagram of reclassification from the KEYNOTE-564 risk classification to the MPRS. b Kaplan‒Meier analysis of the KEYNOTE-564 risk classification for DFS in all enrolled patients. c, d Kaplan–Meier analysis of the MPRS for DFS of patients in the KEYNOTE-564 low-risk, inter and high-risk groups. e Kaplan–Meier analysis for DFS between patients classified as inter&high-risk by the KEYNOTE-564 risk classification but as low-risk by the MPRS, and patients classified as low-risk by the KEYNOTE-564 risk classification but as high-risk by the MPRS. p values were calculated with the log-rank test. MPRS=multimodal predictive recurrence score. DFS = disease-free survival.

Fig. 4. Interpretability analysis of the model.

a SHAP values for clinical features that drive the prediction of recurrence in the model for ccRCC. b–e Grad-CAM analysis of images obtained with the radiological and histopathological modalities. The clinical information of representative patients is included above the figure: sex, age, TNM stage, tumor grade, risk group defined by our model, and clinical outcome (recurrence status). On the basis of the importance scores, the deep red areas suggest greater contributions to the prediction of tumor recurrence, and the deep blue areas indicate lower contributions. Radiological modality: High-attention regions (deep red) localized predominantly in the irregular border area of the tumor (b) and the heterogeneous area of enhancement with a rich blood supply and necrosis (c). Histopathological modality: The model focused more on sarcomatoid differentiated tissue (Regions 1 and 3, deep red) than on well-differentiated tissue (Region 2, light blue) and high-grade tissue (Region 4, yellow‒green) (d); comparatively, it placed greater emphasis on poorly differentiated, higher-grade tissue (Regions 2 and 3, deep red) than on stroma-rich tissue (Region 1, deep blue) and well-differentiated, lower-grade tissue (Region 4, yellow‒green) (e).

Fig. 5. Comparative analyses of MPRS and clinical risk stratification tools in the combined internal and external validation cohorts.

a–d Confusion matrices for MPRS, KEYNOTE-564 risk classification, Leibovich score, and UISS score in prediction of recurrence risk stratification. e, f Heatmaps for comparative analysis of recurrence risk prediction between MPRS and KEYNOTE-564 risk classification, Leibovich score, and UISS score. TP = true positive. FN = false negative. TN = true negative. FP = false positive.

Fig. 6. Analysis of representative cases of misclassification.

Grad-CAM analysis of images obtained with radiological and histopathological modalities. The clinical information of representative patients is included above the figure: gender, age, TNM stage, tumor grade, risk group defined by our model, and clinical outcome (recurrence status). On the basis of the importance scores, the deep red areas suggest greater contributions to the prediction of tumor recurrence, and the deep blue areas indicate lower contributions. a Radiological modality: High-attention regions (deep red) localize primarily in the tumor mass, while areas with renal vein tumor thrombus (yellow–green) receive insufficient attention, indicating that the model failed to focus on capturing the tumor thrombus area. b Radiological modality: Central necrotic areas (deep red) dominate attention maps, with peripheral enhancing tumor regions (yellow–green) being overlooked, revealing an inadequate model balance between necrosis and peripheral enhancing tumor regions. c Histopathological modality: Our model focused more on normal renal tissue (Region 4, deep red) than on tumor tissue (Regions 1-3, yellow–green). d Histopathological modality: Our model primarily identified the tumor tissue region (Region 1, deep red) but missed the necrotic tissue areas (Regions 2–4, deep blue).