Significant adverse prognostic events in patients with urosepsis: a machine learning based model development and validation study

Abstract

Background: Urosepsis is a subset of sepsis with a high mortality rate. Currently, the ranking of urosepsis in sepsis etiology is on the rise. Our goal is to use machine learning (ML) methods to construct and validate an interpretable prognosis prediction model for patients with urosepsis.

Method: Data were collected from the Intensive Care Medical Information Mart IV database version 3.1 and divided into a training cohort and a validation cohort in a 7:3 ratio. Random Forest (RF), Lasso, Boruta, and eXtreme Gradient Boosting (XGBoost) were used to identify the most influential variables in the model development dataset, and the optimal variables were selected based on achieving the λ_1se value. Model development includes seven machine learning methods and ten cross validations. Accuracy and Decision Curve Analysis (DCA) were used to evaluate the performance of the model in order to select the optimal model. Internal validation of the model included area under the ROC curve (AUC), sensitivity, specificity, Matthews correlation coefficient, and F1-score. Finally, SHapley Additive exPlans (SHAP) was used to explain ML models.

Result: A total of 1389 patients with urosepsis were included. Optimal predictors were selected through statistical regularization, yielding a parsimonious set of 9 variables for model development. The performance of XGBoost model is the best and the accuracy of XGBoost was 0.818, with an AUC of 0.904 (95% CI: 0.886-0.923). The internal validation accuracy was 0.797, AUC was 0.869 (95% CI: 0.834-0.904), sensitivity was 0.797, specificity was 0.752, Matthews correlation coefficient was 0.597, and F1-score was 0.791. This indicates that the predictive model performs well in internal validation. SHAP-based summary graphs and diagrams were used to globally explain the XGBoost model.

Conclusion: ML demonstrates strong prognostic capability in urosepsis, with the SHAP method providing clinically intuitive explanations of model predictions. This enables clinicians to identify critical prognostic factors and personalize treatments. While our model achieved high predictive accuracy, its retrospective derivation from a single-center database necessitates external validation in diverse populations, which should be addressed through future prospective multicenter studies to establish clinical generalizability.

Keywords: MIMIC-IV database; SHAP; machine learning; prognostic model; urosepsis.

Figure 1

Study cohort selection and model development workflow. From 58,078 urinary tract infection patients in MIMIC-IV, 1,389 met inclusion criteria. Twelve key features were selected using four machine learning methods (RF, Lasso, Boruta, XGBoost), refined to nine variables through clinical review. The cohort was split 7:3 (training: testing). Seven ML models were trained; XGBoost demonstrated optimal performance and was validated.

Figure 2

Variable Wayne diagram screened by four methods. The important variables and their intersection relationships selected by four feature selection methods (Boruta, XGBoost, Random Forest (RF), and Lasso) are presented in the form of Venn diagrams. The different colored blocks in the figure represent different methods and the intersection of color blocks represents the important variables selected by different methods together.

Figure 3

The variables selected by the four methods were sorted by importance. (A) RF; (B) Lasso; (C) Boruta; (D) XGBoost;.

Figure 4

Comparison of ROC curves of seven models in the training cohort. Red line =Bayes model, orange line = DT model, green line = LR model, blue line = MLP model, dark purple line = RF model, bright purple line = SVM model, yellow line = XGBoost model.

Figure 5

Comparison of ROC curves of seven models in the internal validation cohort. Red line = Bayes model, orange line = DT model, green line =LR model, blue line =MLP model, dark purple line =RF model, bright purple line = SVM model, yellow line = XGBoost model.

Figure 6

Comparison of the performance of the seven models in the training cohort. Bayes, Bayesian Network; DT, Decision tree; LR, Logistic regression model; MLP, Multilayer perceptron; RF, Random Forest model; SVM, Support vector machine; XGBoost, eXtreme Gradient Boosting.

Figure 7

Decision curve analysis (DCA) of seven prediction models. The net benefit curve of the prognostic model was shown. The x-axis represents the threshold probability of intensive care outcome, and the y-axis represents the net benefit. Red line = Bayes model, orange line = DT model, green line = LR model, blue line = MLP model, dark purple line = RF model, bright purple line = SVM model, yellow line = XGBoost model, black line = Treat all, gray line = Treat none.

Figure 8

The calibration curve plot of the seven models. Red line = Bayes model, orange line = DT model, green line = LR model, blue line = MLP model, dark purple line = RF model, bright purple line = SVM model, yellow line = XGBoost model.

Figure 9

The SHAP method is used to analyze the important features of the XGBoost model. Create a point for each feature attribute value of each patient’s model, thereby assigning a point to each patient on the line for each feature. Dots are colored according to the eigenvalues of the corresponding patients and accumulate vertically to depict the density. Purple indicates high eigenvalues (death in this case), while yellow indicates low eigenvalues. The farther the point is from the baseline SHAP value, the greater the impact on the output.

Figure 10

Local model explanation by the Shapley Additive Explanations (SHAP) method. (A) Non-survival patient. (B) Survival patient. Each patient is represented by the x-axis, while the feature contribution is represented by the y-axis: an increased red part for each individual patient represents a greater probability toward the decision of “Non-survival”.