Machine-learning-derived online prediction models of outcomes for patients with cholelithiasis-induced acute cholangitis: development and validation in two retrospective cohorts

Abstract

Background: Cholelithiasis-induced acute cholangitis (CIAC) is an acute inflammatory disease with poor prognosis. This study aimed to create machine-learning (ML) models to predict the outcomes of patients with CIAC.

Methods: In this retrospective cohort and ML study, patients who met the both diagnosis of 'cholangitis' and 'calculus of gallbladder or bile duct' according to the International Classification of Disease (ICD) 9th revision, or met the diagnosis of 'calculus of bile duct with acute cholangitis with or without obstruction' according to the ICD 10th revision during a single hospitalization were included from the Medical Information Mart for Intensive Care database, which records patient admissions to Beth Israel Deaconess Medical Center, MA, USA, spanning June 1, 2001 to November 16, 2022. Patients who were neither admitted in an emergency department nor underwent biliary drainage within 24 h after admission, had an age of less than 18, or lost over 20% of the information were excluded. Nine ML methods, including the Logistic Regression, eXtreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine, Adaptive Boosting, Decision Tree, Gradient Boosting Decision Tree, Gaussian Naive Bayes, Multi-Layer Perceptron, and Support Vector Machine were applied for prediction of in-hospital mortality, re-admission within 30 days after discharge, and mortality within 180 days after discharge. Patients from Zhongda Hospital affiliated to Southeast University in China between January 1, 2019 and July 30, 2023 were enrolled as an external validation set. The area under the receiver operating characteristic curve (AUROC) was the main index for model performance assessment.

Findings: A total of 1156 patients were included to construct models. We performed stratified analyses on all patients, patients admitted to the intensive care unit (ICU) and those who underwent biliary drainage during ICU treatment. 13-16 features were selected from 186 variables for model training. The XGBoost method demonstrated the most optimal predictive efficacy, as evidenced by training set AUROC of 0.996 (95% CI NaN-NaN) for in-hospital mortality, 0.886 (0.862-0.910) for re-admission within 30 days after discharge, and 0.988 (0.982-0.995) for mortality within 180 days after discharge in all patients, 0.998 (NaN-NaN), 0.933 (0.909-0.957), and 0.988 (0.983-0.993) in patients admitted to the ICU, 0.987 (0.970-0.999), 0.908 (0.873-0.942), and 0.982 (0.971-0.993) in patients underwent biliary drainage during ICU treatment, respectively. Meanwhile, in the internal validation set, the AUROC reached 0.967 (0.933-0.998) for in-hospital mortality, 0.589 (0.502-0.677) for re-admission within 30 days after discharge, and 0.857 (0.782-0.933) for mortality within 180 days after discharge in all patients, 0.963 (NaN-NaN), 0.668 (0.486-0.851), and 0.864 (0.757-0.970) in patients admitted to the ICU, 0.961 (0.922-0.997), 0.669 (0.540-0.799), and 0.828 (0.730-0.925) in patients underwent biliary drainage during ICU treatment, respectively. The AUROC values of external validation set consisting of 61 patients were 0.741 (0.725-0.763), 0.812 (0.798-0.824), and 0.848 (0.841-0.859), respectively.

Interpretation: The XGBoost models could be promising tools to predict outcomes in patients with CIAC, and had good clinical applicability. Multi-center validation with a larger sample size is warranted.

Funding: The Technological Development Program of Nanjing Healthy Commission, and Zhongda Hospital Affiliated to Southeast University, Jiangsu Province High-Level Hospital Construction Funds.

Keywords: Acute cholangitis; Cholelithiasis; MIMIC; Prediction model; Prognosis; XGBoost.

Fig. 1

The overall flowchart of the study (a). The algorithm chart of the study (b). CIAC, cholelithiasis-induced acute cholangitis; MIMIC, Medical Information Mart for Intensive Care; HR, heart rate; RR, respiratory rate; SBP, systolic blood pressure; DBP, diastolic blood pressure; WBC, white blood cell; TBil, total bilirubin; SOFA, sepsis-related organ failure; GCS, glasgow coma scale; ICD, International Classification of Disease.

Fig. 2

Feature selection based on the SelectFromModel algorithm. The horizontal axis represents the name of each variable, while the vertical axis represents the significance attributed to each variable. Outcomes of all the patients (a–c): in-hospital mortality, re-admission within 30 days after discharge and mortality within 180 days after discharge. Outcomes of patients admitted to the ICU (d–f). Outcomes of patients underwent biliary drainage during ICU treatment (g–i). ICU, intensive care unit; PTT, partial thromboplastin time; RDW, red blood cell distribution width; LD, lactate dehydrogenase; WBC, white blood cells; TBil, total bilirubin; ALT, alanine aminotransferase; RBC, red blood cells; SOFA, sepsis-related organ failure; LODS, logistic organ dysfunction system; APSIII, acute physiology score iii; ALP, alkaline phosphatase; MCHC, mean corpuscular hemoglobin concentration; AKI, acute kidney injury; OASIS, oxford acute severity of illness score.

Fig. 3

Receiver operating characteristic curve and Precision-Recall curve of nine models. Outcomes of all the patients: in-hospital mortality (a–c), re-admission within 30 days after discharge (d–f), and mortality within 180 days after discharge (g–i). XGBoost, eXtreme Gradient Boosting; LightBGM, Light Gradient Boosting Machine; AdaBoost, Adaptive Boosting; GBDT, Gradient Boosting Decision Tree; GNB, Gaussian Naive Bayes; MLP, Multi-Layer Perceptron; SVM, Support Vector Machine.

Fig. 4

The decision curve analyses of XGBoost and the model with the most disputable predictive efficacy apart from XGBoost. Outcomes of all the patients: in-hospital mortality (a), re-admission within 30 days after discharge (b), and mortality within 180 days after discharge (c). XGBoost, eXtreme Gradient Boosting; AdaBoost, Adaptive Boosting.

Fig. 5

An example of web tool usage. The prediction of in-hospital mortality was conducted by inputting 15 clinical parameters pertaining to a single patient underwent biliary drainage during ICU treatment, thereby indicating an unfavorable prognosis.