A predictive machine-learning model for clinical decision-making in washed microbiota transplantation on ulcerative colitis

Abstract

Machine learning based on clinical data and treatment protocols for better clinical decision-making is a current research hotspot. This study aimed to build a machine learning model on washed microbiota transplantation (WMT) for ulcerative colitis (UC), providing patients and clinicians with a new evaluation system to optimize clinical decision-making. Methods Patients with UC who underwent WMT via mid-gut or colonic delivery route at an affiliated hospital of Nanjing Medical University from April 2013 to June 2022 were recruited. Model ensembles based on the clinical indicators were constructed by machine-learning to predict the clinical response of WMT after one month. Results A total of 366 patients were enrolled in this study, with 210 patients allocated for training and internal validation, and 156 patients for external validation. The low level of indirect bilirubin, activated antithrombin III, defecation frequency and cholinesterase and the elderly and high level of creatine kinase, HCO₃ ^- and thrombin time were related to the clinical response of WMT at one month. Besides, the voting ensembles exhibited an area under curve (AUC) of 0.769 ± 0.019 [accuracy, 0.754; F1-score, 0.845] in the internal validation; the AUC of the external validation was 0.614 ± 0.017 [accuracy, 0.801; F1-score, 0.887]. Additionally, the model was available at https://wmtpredict.streamlit.app. Conclusions This study pioneered the development of a machine learning model to predict the one-month clinical response of WMT on UC. The findings demonstrate the potential value of machine learning applications in the field of WMT, opening new avenues for personalized treatment strategies in gastrointestinal disorders. Trial registration clinical trials, NCT01790061. Registered 09 February 2013 - Retrospectively registered, https://clinicaltrials.gov/study/NCT01790061.

Keywords: Clinical indicator; Fecal microbiota transplant; Machine learning; Ulcerative colitis.

Graphical abstract

Fig. 1

Flow chart of the study. Framework for dataset partition, model training, independent validation (internal and external verification), and the online application. The individuals were divided into two groups, including the effective (green) and ineffective (red), according the efficacy at one month after WMT. LR, logistic regression; RF, random forests; Ada, adaptive boosting; LGB, light gradient boosting machine; SVM, support vector machine; MLP, multi-layer perceptron; VTM, voting machine.

Fig. 2

The evaluation of Model ensembles on test data of internal verification. (A) The evaluation of Model ensemble A on test data of internal verification. (B) The evaluation of Model Ensemble B on test data of internal verification. (C) The evaluation of Model Ensemble C on test data of internal verification. ROC curve, receiver operating characteristic curve; AUC, the area under curve of receiver operating characteristic; LGBM, light gradient boosting machine; MLP, multi-layer perceptron.

Fig. 3

The evaluation on generalization ability of model ensembles for external verification. (A) The evaluation on Model ensemble A of external verification. (B) The evaluation on Model Ensemble B of external verification. (C) The evaluation on Model Ensemble C of external verification. (D) The comparison of AUC score in different ensembles. (E) The comparison of accuracy in different ensembles. (F) The comparison of F1-score in different ensembles. (G) The comparison of brier score in different ensembles. (H) The comparison of AUC score in different algorithms. ROC curve, receiver operating characteristic curve; AUC, the area under curve of receiver operating characteristic; LR, logistic regression; RF, random forest; LGBM, light gradient boosting machine; MLP, multi-layer perceptron; VTM, voting machine; SVM, support vector machine. * , P < 0.05; * *, P < 0.01; * ** , P < 0.001; P-value < 0.05 was considered as statistically significant.

Fig. 4

The explanation and feature importance of model ensembles on external verification. (A) The summary plot of shap value on the voting machine of model ensemble A based on external verification. (B) The summary plot of shap value on the voting machine of model Ensemble B based on external verification. (C) The summary plot of shap value on the voting machine of model Ensemble C based on external verification. The importance of feature decreased from top to bottom. And the color of the points represented the value of corresponding variable every sample. The color of the points ranged from blue to red, the value ranged from low to high. The location of the point on the horizontal axis represented the shap value, which was positive to the possibility of clinical response of WMT after one month. P-LCR, platelet-larger cell ratio; PDW, platelet distribution width; GGT, γ-glutamyl transpeptidase; IB, indirect bilirubin or unconjugated bilirubin; CHE, cholinesterase; CK, creatine kinase; TT, thrombin time; AT-IIIa, activates antithrombin III; INR, international normalized ratio.