Multimodal deep learning integration for predicting renal function outcomes in living donor kidney transplantation: a retrospective cohort study

Abstract

Background: Accurately predicting post-transplant renal function is essential for optimizing donor-recipient matching and improving long-term outcomes in kidney transplantation (KT). Traditional models using only structured clinical data often fail to account for complex biological and anatomical factors. This study aimed to develop and validate a multimodal deep learning model that integrates computed tomography (CT) imaging, radiology report text, and structured clinical variables to predict 1-year estimated glomerular filtration rate (eGFR) in living donor kidney transplantation (LDKT) recipients.

Materials and methods: A retrospective cohort of 1,937 LDKT recipients was selected from 3772 KT cases. Exclusions included deceased donor KT, immunologic high-risk recipients ( n = 304), missing CT imaging, early graft complications, and anatomical abnormalities. eGFR at 1 year post-transplant was classified into four categories: >90, 75-90, 60-75, and 45-60 mL/min/1.73 m 2 . Radiology reports were embedded using BioBERT, while CT videos were encoded using a CLIP-based visual extractor. These were fused with structured clinical features and input into ensemble classifiers including XGBoost. Model performance was evaluated using cross-validation and SHapley Additive exPlanations (SHAP) analysis.

Results: The full multimodal model achieved a macro F1 score of 0.675, micro F1 score of 0.704, and weighted F1 score of 0.698 - substantially outperforming the clinical-only model (macro F1 = 0.292). CT imaging contributed more than text data (clinical + CT macro F1 = 0.651; clinical + text = 0.486). The model showed highest accuracy in the >90 (F1 = 0.7773) and 60-75 (F1 = 0.7303) categories. SHAP analysis identified donor age, BMI, and donor sex as key predictors. Dimensionality reduction confirmed internal feature validity.

Conclusion: Multimodal deep learning integrating clinical, imaging, and textual data enhances prediction of post-transplant renal function. This framework offers a robust and interpretable approach for individualized risk stratification in LDKT, supporting precision medicine in transplantation.

Keywords: kidney transplant; machine learning; multimodal deep learning; post-transplant outcome prediction; prognosis.

Figure 1.

Study cohort flow diagram for model development and analysis.

Figure 2.

Overview of the proposed multimodal classification framework. The model integrates structured clinical variables, radiology report text, and CT video data. Clinical information is processed as tabular features; radiology text is embedded using a pretrained BERT model; and CT video frames are encoded via a CLIP-based image encoder. Each modality undergoes feature extraction and mean pooling, followed by feature concatenation and min–max normalization. The final fused representation is input into an ML classifier (xGBoost/RF/SVM/LR) to predict the post-transplantation eGFR category.

Figure 3.

Visualization of multimodal feature embeddings using two-dimensional projection techniques. The fused embeddings obtained from structured, textual, and visual features were reduced to two dimensions using t-distributed stochastic neighbor embedding (t-SNE) (A) and uniform manifold approximation and projection (UMAP) (B) Each point represents a patient sample, color-coded by the corresponding post-transplantation eGFR class (>90, 75–90, 60–75, or 45–60). While both methods reveal local grouping tendencies, substantial overlap among classes is observed. T-SNE emphasizes local clusters within classes, whereas UMAP preserves more of the global structure, enabling visualization of broader inter-class relationships.

Figure 4.

Multiclass classification performance of final model using XGBoost. Confusion matrix summarizing the performance of the final multimodal model trained with XGBoost across four post-transplant eGFR categories: > 90, 75–90, 60–75, and 45–60 mL/min/1.73 m². Precision, recall, and F1-scores are reported for each class. The model achieved the highest precision and recall for the >90 and 60–75 categories, with lower performance for the 45–60 group due to its smaller sample size. Overall macro and weighted F1-scores were 0.675 and 0.698, respectively, indicating strong class-balanced performance. Most misclassifications occurred between adjacent eGFR strata, reflecting clinical overlaps in renal function.

Figure 5.

Mean absolute SHAP values for all features, color-coded by class. Donor age (Dage), BMI, and donor sex (Dsex) were consistently influential across all classes. This multi-class SHAP summary reveals both shared and class-specific key predictors.

Figure 6.

Class-specific SHAP summary plots for predicting each eGFR category: (A) > 90, (B) 75–90, (C) 60–75, and (D) 45–60. Each plot displays the top contributing features for the corresponding class, with SHAP values representing the magnitude and direction of each feature’s influence on the model output. While certain features such as Dage, BMI, and D_cr consistently appear across classes, other features demonstrate class-specific importance. For example, Dsex and AGE were more predictive of Class (A), whereas DM and DBMI were prominent in Class (B), and Induction appeared more influential in Class (D). These findings highlight both shared and distinct patterns in multimodal predictors of kidney function.