Establishment of predictive models for postoperative delirium in elderly patients after knee/hip surgery based on total bilirubin concentration: machine learning algorithms

Abstract

Background: With the aging demographic on the rise, we're seeing a spike in the occurrence of postoperative delirium (POD). Our research aims to delve into the connection between plasma bilirubin levels and postoperative delirium, with the goal of crafting ten machine learning (ML) models capable of predicting POD instances.

Methods: This study enrolled 621 elderly patients after knee/hip surgery. We used the Confusion Assessment Method (CAM) to assess whether participants had POD. Univariate binary logistic regression analysis and restricted cubic spline (RCS) analysis were used to evaluate the association between plasma total bilirubin and POD. This study further investigated whether cerebrospinal fluid plays some role in the relationship between bilirubin and POD using mediated causal analysis. Subsequently, we employed ten machine learning algorithms to train and develop the predictive models: Logistic Regression (LR), Support Vector Machine (SVM), Gradient Boosting Model (GBM), Neural Network (NN), Random Forest (RF), Xgboost, K-Nearest Neighbors (KNN), AdaBoost, LightGBM, and CatBoost. The performance of the models was evaluated by the area under the receiver operating characteristic curve (AUROC), Brier score, accuracy, sensitivity, specificity, precision, F1 score, calibration curve, decision curve, clinical impact curve, and confusion matrix. In addition, the model was interpreted through Shapley additive interpretation (SHAP) analysis to clarify the importance of bilirubin in the model and its decision-making basis.

Results: Univariate binary logistic regression analysis revealed that plasma total bilirubin was associated with POD. Furthermore, the RCS analysis illustrated there was no nonlinear relationship between total bilirubin and POD. Mediation analysis indicted that T-tau mediated the effect of total bilirubin on POD. Total bilirubin and other features(age, educational level, BMI, history of diabetes, ASA, albumin, Aβ42, T-tau and P-tau) were used to construct ML models. Compared with other ML algorithms, NN showed better performance, with an AUC of 0.973 (95% CI (0.959-0.987)) in the test set. In addition, the SHAP method determines that age and education are the main determinants that affect the prediction of ML models.

Conclusion: Plasma total bilirubin was identified as a preoperative risk factor for postoperative delirium (POD). Among ten ML models, the Neural Network (NN) incorporating total bilirubin showed the best predictive performance for POD.

Trial registration: Clinical Registration No. ChiCTR2000033439. Registration data:2020.06.01.

Keywords: Cerebrospinal fluid; Machine learning; Postoperative delirium; Surgery.

Fig. 1

Delirium assessment

Fig. 2

Flowchart of our study procedure

Fig. 3

(A) The box-plots, box-plots show that the level of plasma total bilirubin in the postoperative delirium (POD) group and non-postoperative delirium (NPOD) group. (B) Restricted cubic spline analysis, total bilirubin levels are not non-linearly related to postoperative delirium using a Restricted Cubic Spline Analysis. (C) Mediation analysis. Mediation analysis mediates the relationship between total bilirubin and POD

Fig. 4

(A) Receiver operating characteristic curves for ten machine learning models in training dataset. (B) Receiver operating characteristic curves for ten machine learning models in testing dataset. (C) Calibration plot for ten machine learning models in training dataset. (D) Calibration plot for ten machine learning models in test dataset

Fig. 5

Evaluation the performance of all machine learning models in training dataset and the test dataset

Fig. 6

(A) Decision curve analysis for NN model in training dataset. (B) Decision curve analysis for NN model in test dataset. (C) Clinical impact curve for NN model in training dataset. (D) Clinical impact curve for NN model in test dataset. (E) Fuzzy matrix for NN model in training dataset. (F) Fuzzy matrix for NN model in test dataset

Fig. 7

(A) SHAP summary plot of the NN model. The color represents the value of the variable. (B) SHAP importance for each feature of the NN model. (C) SHAP force plot of the NN model.