Machine Learning-Based Prediction of Early Complications Following Surgery for Intestinal Obstruction: Multicenter Retrospective Study

Abstract

Background: Early complications increase in-hospital stay and mortality after intestinal obstruction surgery. It is important to identify the risk of postoperative early complications for patients with intestinal obstruction at a sufficiently early stage, which would allow preemptive individualized enhanced therapy to be conducted to improve the prognosis of patients with intestinal obstruction. A risk predictive model based on machine learning is helpful for early diagnosis and timely intervention.

Objective: This study aimed to construct an online risk calculator for early postoperative complications in patients after intestinal obstruction surgery based on machine learning algorithms.

Methods: A total of 396 patients undergoing intestinal obstruction surgery from April 2013 to April 2021 at an independent medical center were enrolled as the training cohort. Overall, 7 machine learning methods were used to establish prediction models, with their performance appraised via the area under the receiver operating characteristic curve (AUROC), accuracy, sensitivity, specificity, and F₁-score. The best model was validated through 2 independent medical centers, a publicly available perioperative dataset the Informative Surgical Patient dataset for Innovative Research Environment (INSPIRE), and a mixed cohort consisting of the above 3 datasets, involving 50, 66, 48, and 164 cases, respectively. Shapley Additive Explanations were measured to identify risk factors.

Results: The incidence of postoperative complications in the training cohort was 47.44% (176/371), while the incidences in 4 external validation cohorts were 34% (17/50), 56.06% (37/66), 52.08% (25/48), and 48.17% (79/164), respectively. Postoperative complications were associated with 8-item features: Physiological Severity Score for the Enumeration of Mortality and Morbidity (POSSUM physiological score), the amount of colloid infusion, shock index before anesthesia induction, ASA (American Society of Anesthesiologists) classification, the percentage of neutrophils, shock index at the end of surgery, age, and total protein. The random forest model showed the best overall performance, with an AUROC of 0.788 (95% CI 0.709-0.869), accuracy of 0.756, sensitivity of 0.695, specificity of 0.810, and F₁-score of 0.727 in the training cohort. The random forest model also achieved a comparable AUROC of 0.755 (95% CI 0.652-0.839) in validation cohort 1, a greater AUROC of 0.817 (95% CI 0.695-0.913) in validation cohort 2, a similar AUROC of 0.786 (95% CI 0.628-0.902) in validation cohort 3, and the comparable AUROC of 0.720 (95% CI 0.671-0.768) in validation cohort 4. We visualized the random forest model and created a web-based online risk calculator.

Conclusions: We have developed and validated a generalizable random forest model to predict postoperative early complications in patients undergoing intestinal obstruction surgery, enabling clinicians to screen high-risk patients and implement early individualized interventions. An online risk calculator for early postoperative complications was developed to make the random forest model accessible to clinicians around the world.

Keywords: Shapley additive explanations; early intervention; intestinal obstruction; machine learning; postoperative complications; prediction model; risk calculator.

Figure 1

Flowchart of patient enrollment. ADA: adaptive boosting classifier; GBM: gradient boosting machine; GNB: Gaussian Bayesian classifier; LR: logistic regression; MLP: multilayer perceptron; RF: random forest; SVM: support vector machine.

Figure 2

The process of using a wrapper feature selection method to screen features. Upon the initial selection of the first 8 features, the model achieved its peak area under the receiver operating characteristic (AUROC) curve on the validation set. Subsequent addition of features did not result in a further increase of the AUROC. Consequently, we opted for the top 8 features with the highest Shapley Additive Explanations value.

Figure 3

Use of the receiver operating characteristic curve plots to demonstrate the performance of the machine learning model in different cohorts. ADA: adaptive boosting; GBM: gradient boosting machine; GNB: Gaussian Bayesian classifier; LR: logistic regression; MLP: multilayer perceptron; RF: random forest; SVM: support vector machine.

Figure 4

Use of Shapley Additive Explanations (SHAP) summary plot and SHAP dependence plot to demonstrate the importance and impact of variables on the random forest model. ASA: American Society of Anesthesiologists; SHAP: Shapley Additive Explanations; NEUT%: neutrophil percentage; POSSUM: Physiological and Operative Severity Score for the Enumeration of Mortality and Morbidity.

Figure 5

Use the SHAP decision plots and SHAP force plots to demonstrate the decision pathways and feature importance of the RF model in specific patients.

Figure 6

Calculation tool of random forest model to estimate early complications after intestinal obstruction surgery. After inputting 8 variables into the random forest model, the prediction model output for the case was “positive” with a probability of 86%.