Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Jul 29:12:1624198.
doi: 10.3389/fmed.2025.1624198. eCollection 2025.

Explainable machine learning for predicting distant metastases in renal cell carcinoma patients: a population-based retrospective study

Zhao Hou  1   2 , Peipei Wang  1   3 , Dingyang Lv  1   4 , Huiyu Zhou  1   4 , Zhiwei Guo  1   4 , Jinshuai Li  1   4 , Mohan Jia  1   4 , Hongyang Du  1   4 , Weibing Shuang  1   4
Affiliations

Explainable machine learning for predicting distant metastases in renal cell carcinoma patients: a population-based retrospective study

Zhao Hou et al. Front Med (Lausanne). .

Abstract

Background: Distant metastasis is a key factor contributing to poor prognosis in renal cell carcinoma (RCC). Early prediction of metastasis is crucial for developing personalized treatment plans and improving patient outcomes. This study aimed to establish and validate a clinical prediction model for distant metastasis in RCC patients.

Methods: Ten machine learning algorithms were employed to develop a predictive model for distant metastasis in RCC. Data from 51,566 RCC patients in The Surveillance, Epidemiology, and End Results (SEER) database (2010-2018) were used for model development, while 726 RCC patients from the First Hospital of Shanxi Medical University were selected for external validation. Hyperparameters were optimized using grid search and tenfold cross-validation. Model performance was assessed using metrics such as the area under the receiver operating characteristic curve (AUC), the area under the precision-recall curve (AUPRC), decision curve analysis, calibration curves, precision, and accuracy. Shapley additive explanations (SHAP) were used for model interpretation. The best-performing model was then used to create a web-based calculator to predict metastasis risk in RCC patients.

Results: The study included 51,566 RCC patients, with 3,667 showing distant metastases. Logistic regression identified tumor size, grade, T-stage, N-stage, radiotherapy, chemotherapy, and surgery as independent risk factors. The Extreme Gradient Boosting (XGB) model demonstrated superior performance (AUC: 0.957, Accuracy: 0.898) in the training set and was validated externally (AUC: 0.742, Accuracy: 0.904). A web-based calculator was developed using the XGB model.

Conclusion: This study designed and validated an XGB model using clinicopathologic data to predict the risk of distant metastasis in RCC patients, potentially aiding clinical decision-making.

Keywords: distant metastasis; external validation; machine learning; predictive modeling; renal cell carcinoma; web-based calculator.

PubMed Disclaimer

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figures

Flowchart detailing the selection and processing of renal cell carcinoma patient data from the SEER database (2010-2018). Out of 347,786 patients, 51,566 were included; 296,220 were excluded due to inadequate data. The dataset was split into a training set of 36,096 and a testing set of 15,470 in a 7:3 ratio. Various machine learning models were applied, including LR, DT, RF, NBC, KNN, SVM, Enet, MLP, XGB, and LghtGBM, using k-fold cross-validation with stochastic optimization. The best model was validated with 726 external patients from Shanxi Medical University for constructing a web calculator.
FIGURE 1
Study design and patient screening workflow diagram.
Chart (A) shows a line plot of coefficients vs. log lambda values ranging from -8 to 1, with colored lines representing different coefficients. Chart (B) displays a scatter plot of binomial deviance vs. log lambda, with red dots indicating data points showing an upward trend from log(lambda) = -6 to -2.
FIGURE 2
Risk factors for distant metastasis of renal cancer identified by LASSO regression. (A) Based on the logarithmic (lambda) sequence, a coefficient profile was created, yielding non-zero coefficients corresponding to the optimal lambda value. (B) The process of selecting the optimal value for the parameter λ in the Lasso regression model was performed using cross-validation. The dotted vertical lines indicate the optimal predictors based on the minimum criteria and the 1 standard error of the minimum criteria (1-SE criteria).
Graphs compare eleven models (LR, ENet, DT, RF, XGB, SVM, MLP, LightGBM, KNN, NBC) across three panels (A, B, C) plotting percent predictions against midpoints. Each model’s performance varies, showing red lines against a benchmark diagonal.
FIGURE 3
Calibration curves of 10 machine learning methods in the training set (A), test set (B), and external validation set (C). The black diagonal line represents the ideal calibration curve. A calibration curve closer to this line indicates better model calibration.
Four-panel image with different machine learning model evaluations. (A) ROC curve showing sensitivity vs. 1-specificity for multiple models: DT, ENet, KNN, LightGBM, LR, MLP, NBC, RF, SVM, XGB. (B) Precision-recall curve for the same models. (C) Decision curve analysis with net benefit vs. threshold probability for various strategies, including Treat All and Treat None. (D) Calibration plot for the XGB model showing predicted probability vs. observed outcomes across midpoints.
FIGURE 4
The receiver operating characteristic (ROC) curves (A), Precision-Recall (PR) curves (B), Decision Curve Analysis (DCA) curves (C), and calibration curves (D) of the 10 machine learning models in the training set, with calibration curves based on the best model.
Panel (A) displays an ROC curve comparing model sensitivity and 1-specificity for different models including DT, ENet, KNN, and others. Panel (B) shows a precision-recall curve for the same models. Panel (C) presents a decision curve analysis plotting net benefit against threshold probability. Panel (D) is a calibration curve for the XGB model, displaying the percent against the midpoint. Each panel includes detailed legends for model identification.
FIGURE 5
The receiver operating characteristic (ROC) curves (A), Precision-Recall (PR) curves (B), Decision Curve Analysis (DCA) curves (C), and calibration curves (D) of the 10 machine learning models in the test set, with calibration curves based on the best model.
Four plots are displayed: (A) Receiver Operating Characteristic (ROC) curves comparing models like DT, KNN, and XGB; (B) Precision-Recall curves for the same models; (C) Net Benefit against Threshold Probability for various models; (D) Calibration plot for the XGB model with percent on the y-axis and midpoint on the x-axis.
FIGURE 6
The receiver operating characteristic (ROC) curves (A), Precision-Recall (PR) curves (B), Decision Curve Analysis (DCA) curves (C), and calibration curves (D) of the 10 machine learning models in the external validation set, with calibration curves based on the best model.
Three heatmaps labeled A, B, and C compare model performance across five metrics: Accuracy, AUC, F2-Score, Precision, and Recall. Each heatmap shows different models like XGB, SVM, and RF. Darker shades indicate higher values, while lighter shades represent lower values. Heatmap A shows generally high values across metrics, B has moderate values, and C presents the lowest metrics, particularly in Recall.
FIGURE 7
Predictive performance of 10 models in the training set (A), test set (B), and external validation set (C).
Panel A shows a violin plot of SHAP values for different features. Higher values are in purple and lower values in yellow, indicating feature impact intensity. Panel B displays a bar chart ranking the same features by importance score, with Chemotherapy_X1 having the highest score.
FIGURE 8
Relative importance of variables based on SHAP for XGB prediction model. Where (A) illustrates the SHAP value distribution of features and (B) shows the feature importance scores visualized as a bar plot.Chemotherapy_X1 indicates receipt of chemotherapy, Radiation_X1 indicates receipt of radiotherapy, T_X3 represents tumor stage T3, size_X2 indicates a tumor size greater than 5 cm, RX_X1 indicates receipt of surgical treatment, Grade_X4 represents tumor grade IV, Grade_X3 represents tumor grade III, T_X4 represents tumor stage T4, N_X1 represents tumor stage N1, T_X2 represents tumor stage T2, N_X2 represents tumor stage N2, and Grade_X2 represents tumor grade II. SHAP, Shapley’s Additive Explanation; RX, RX Summ-Surg (surgery).
Distant Metastasis Predictor interface for Renal Cell Carcinoma showing input fields and a prediction result. Inputs: Tumor size ≤5 cm, Grade I, T_stage T1, N_stage N0, Radiotherapy No, Chemotherapy Yes, Surgery Yes. Prediction result indicates a 54.66% probability of distant metastasis, categorized as high risk.
FIGURE 9
An online web-based calculator for predicting distant metastasis of renal cell carcinoma.
See this image and copyright information in PMC

References

    1. Powles T, Albiges L, Bex A, Comperat E, Grünwald V, Kanesvaran R, et al. Renal cell carcinoma: esmo Clinical Practice Guideline for diagnosis, treatment and follow-up. Ann Oncol. (2024) 35:692–706. 10.1016/j.annonc.2024.05.537 - DOI - PubMed
    1. Siegel R, Giaquinto A, Jemal A. Cancer statistics, 2024. CA Cancer J Clin. (2024) 74:12–49. 10.3322/caac.21820 - DOI - PubMed
    1. Bukavina L, Bensalah K, Bray F, Carlo M, Challacombe B, Karam J, et al. Epidemiology of renal cell carcinoma: 2022 Update. Eur Urol. (2022) 82:529–42. 10.1016/j.eururo.2022.08.019 - DOI - PubMed
    1. Escudier B, Porta C, Schmidinger M, Rioux-Leclercq N, Bex A, Khoo V, et al. Renal cell carcinoma: esmo Clinical Practice Guidelines for diagnosis, treatment and follow-up. Ann Oncol. (2019) 30:706–20. 10.1093/annonc/mdz056 - DOI - PubMed
    1. Chen Y, Wang L, Panian J, Dhanji S, Derweesh I, Rose B, et al. Treatment landscape of renal cell carcinoma. Curr Treat Options Oncol. (2023) 24:1889–916. 10.1007/s11864-023-01161-5 - DOI - PMC - PubMed

LinkOut - more resources