Development and validation of machine learning models to predict MDRO colonization or infection on ICU admission by using electronic health record data

Abstract

Background: Multidrug-resistant organisms (MDRO) pose a significant threat to public health. Intensive Care Units (ICU), characterized by the extensive use of antimicrobial agents and a high prevalence of bacterial resistance, are hotspots for MDRO proliferation. Timely identification of patients at high risk for MDRO can aid in curbing transmission, enhancing patient outcomes, and maintaining the cleanliness of the ICU environment. This study focused on developing a machine learning (ML) model to identify patients at risk of MDRO during the initial phase of their ICU stay.

Methods: Utilizing patient data from the First Medical Center of the People's Liberation Army General Hospital (PLAGH-ICU) and the Medical Information Mart for Intensive Care (MIMIC-IV), the study analyzed variables within 24 h of ICU admission. Machine learning algorithms were applied to these datasets, emphasizing the early detection of MDRO colonization or infection. Model efficacy was evaluated by the area under the receiver operating characteristics curve (AUROC), alongside internal and external validation sets.

Results: The study evaluated 3,536 patients in PLAGH-ICU and 34,923 in MIMIC-IV, revealing MDRO prevalence of 11.96% and 8.81%, respectively. Significant differences in ICU and hospital stays, along with mortality rates, were observed between MDRO positive and negative patients. In the temporal validation, the PLAGH-ICU model achieved an AUROC of 0.786 [0.748, 0.825], while the MIMIC-IV model reached 0.744 [0.723, 0.766]. External validation demonstrated reduced model performance across different datasets. Key predictors included biochemical markers and the duration of pre-ICU hospital stay.

Conclusions: The ML models developed in this study demonstrated their capability in early identification of MDRO risks in ICU patients. Continuous refinement and validation in varied clinical contexts remain essential for future applications.

Keywords: Infection; Intensive care unit; Machine learning; Multidrug-resistant organisms; Predictive modeling.

Fig. 1

Receiver operating characteristic curves for the temporal validation of each model (A: PLAGH-ICU; B: MIMIC-IV). lr Logistic regression, knn K-Nearest Neighbor, svc Support Vector Classifier, rf Random Forest, xgb XGBoost eXtreme Gradient Boosting, mlp Multilayer Perceptron

Fig. 2

Probability calibration curves of ensemble models during temporal validation (A: PLAGH-ICU; B: MIMIC-IV)

Fig. 3

Radar Chart of Feature Importance Rankings for Each Model (A: PLAGH-ICU; B: MIMIC-IV). lr Logistic regression, rf Random Forest, xgb XGBoost eXtreme Gradient Boosting, mlp Multilayer Perceptron

Fig. 4

SHAP analysis (A: PLAGH-ICU; B: MIMIC-IV)