Rapid extended-spectrum beta-lactamase-confirmation by using a machine learning model directly on routine automated susceptibility testing results

Abstract

Objectives: Phenotypical Extended Spectrum β-Lactamase (ESBL)-production is commonly determined using the combination disk diffusion test or gradient test. This requires overnight incubation, prolonging time-to-detection and increasing duration of empirical treatment for patients with infections caused by gram-negative bacteria. To achieve instant confirmation without incubation, we developed a machine learning (ML)-model that predicts phenotypic ESBL-confirmation using Minimum Inhibitory Concentrations from routine automated antimicrobial susceptibility testing (AST)-results.

Methods: Data from the Dutch national laboratory-based surveillance system ISIS-AR collected between 2013 and 2022 from 49 laboratories were used: 178,044 isolates of E. coli (141,576), K. pneumoniae (33,088), and P. mirabilis (3,380) that exhibited resistance to cefotaxime and/or ceftazidime, and had available results of phenotypical ESBL-confirmation testing. We evaluated Logistic Regression, Random Forest and XGBoost models and calculated SHAP-values (SHapley Additive exPlanations) to identify most contributing features. We externally validated models using 5,996 isolates collected in Amsterdam University Medical Centres' between 2013 and 2022.

Results: XGBoost achieved an AUROC (Area Under Receiver Operating Characteristics) of 0.97, a sensitivity of 0.89 and an accuracy of 0.93. The most contributing features were the antibiotics cefotaxime, cefoxitin and trimethoprim for E. coli and K. pneumoniae, and cefuroxime, imipenem and cefotaxime for P. mirabilis. External validation yielded AUROCs of 0.93 (E. coli), 0.89 (K. pneumoniae) and 0.93 (P. mirabilis).

Conclusion: ML-models for prediction of ESBL-production using routine AST-system data achieved high performances. Implementing these models in laboratory practice could shorten time-to-detection. Once deployed, this approach could facilitate widespread screening for phenotypic ESBL-production.

Keywords: ESBL; antimicrobial resistance; bacteria; machine learning; surveillance.

FIGURE 1

Average probability distributions in the test set for each combination of species and algorithm. The average is calculated across all ten folds with the error bars indicating the standard deviation of the mean.

FIGURE 2

Average calibration curves across all 10-fold for each developed model. A perfect calibration represents a theoretical shape of the model without error.

FIGURE 3

Feature importance calculated through SHapley Additive exPlanations (SHAP)-values of the XGBoost models. When the impact on the model output is larger than 0 on the X-axis, the feature value contributes to the prediction of a positive Extended Spectrum β-Lactam (ESBL)-confirmation. When the impact on the model output is lower than 0, the contribution is toward a negative outcome of ESBL-confirmation. The color represents the actual value of the feature at that particular prediction: blue represents a low MIC-value, and red a high MIC-value.

FIGURE 4

Visualization of the workflow from sample collection to ESBL-detection. (a) represents the current situation; (b) represents the proposed approach. In (b) the timeline is shortened due to the instant results from the machine learning model. Consequently, the switch from empirical treatment to definite treatment can be made after 24 h.