Predicting future hospital antimicrobial resistance prevalence using machine learning

Abstract

Background: Predicting antimicrobial resistance (AMR), a top global health threat, nationwide at an aggregate hospital level could help target interventions. Using machine learning, we exploit historical AMR and antimicrobial usage to predict future AMR.

Methods: Antimicrobial use and AMR prevalence in bloodstream infections in hospitals in England were obtained per hospital group (Trust) and financial year (FY, April-March) for 22 pathogen-antibiotic combinations (FY2016-2017 to FY2021-2022). Extreme Gradient Boosting (XGBoost) model predictions were compared to the previous value taken forwards, the difference between the previous two years taken forwards and linear trend forecasting (LTF). XGBoost feature importances were calculated to aid interpretability.

Results: Here we show that XGBoost models achieve the best predictive performance. Relatively limited year-to-year variability in AMR prevalence within Trust-pathogen-antibiotic combinations means previous value taken forwards also achieves a low mean absolute error (MAE), similar to or slightly higher than XGBoost. Using the difference between the previous two years taken forward or LTF performs consistently worse. XGBoost considerably outperforms all other methods in Trusts with a larger change in AMR prevalence from FY2020-2021 (last training year) to FY2021-2022 (held-out test set). Feature importance values indicate that besides historical resistance to the same pathogen-antibiotic combination as the outcome, complex relationships between resistance in different pathogens to the same antibiotic/antibiotic class and usage are exploited for predictions. These are generally among the top ten features ranked according to their mean absolute SHAP values.

Conclusions: Year-to-year resistance has generally changed little within Trust-pathogen-antibiotic combinations. In those with larger changes, XGBoost models can improve predictions, enabling informed decisions, efficient resource allocation, and targeted interventions.

Fig. 1. Mean resistance prevalence and standard deviation per Trust–pathogen–antibiotic.

Distribution of mean resistance prevalence (a) and standard deviation (b) per Trust–pathogen–antibiotic across available financial years (Apr2016–Mar2022 for E. coli and MSSA and Apr2017–Mar2022 for Klebsiella sp. and P. aeruginosa). n = 119 Trusts, however not all Trusts contributed data to each boxplot (Supplementary Table 2). Red (E. coli), green (Klebsiella sp.), blue (MSSA), purple (P. aeruginosa). Note: one point per Trust. Outliers outside of the x-axis scale (>50 left panel, >10 right panel) were truncated. Centre line, median; box limits, upper and lower quartiles; whiskers, 1.5× interquartile range.

Fig. 2. Difference in resistance prevalence between the current and previous financial year and estimated change over the study period.

Distribution of difference in resistance prevalence between the current and previous financial year (a) and LTF estimated change over the study period (b), per pathogen–antibiotic combination across all Trusts and financial years. Percentages of Trust-FYs that have an absolute difference <5%, <7.5% and <10% between the current and the previous financial year are also given by pathogen–antibiotic combination (a), and an absolute LTF estimated change <5%, <7.5% and <10% (b). n = 119 Trusts × 5 FY-to-FY differences (E. coli and MSSA) and 4 FY-to-FY differences (Klebsiella sp. and P. aeruginosa), however not all Trusts contributed data to each boxplot (Supplementary Table 2). Red (E. coli), green (Klebsiella sp.), blue (MSSA), purple (P. aeruginosa). Note: one point per trust year. Distribution split by financial year available in Supplementary Fig. 3. Percentage of trusts with an absolute difference in resistance prevalence <5%, <7.5%, and <10% split by financial year in Supplementary Fig. 4. Outliers outside of x-axis scale (absolute value > 20) were truncated. Centre line, median; box limits, upper and lower quartiles; whiskers, 1.5× interquartile range.

Fig. 3. Predictive performance comparison.

Mean absolute error for prediction on test set (resistance prevalence in FY2021-2022) for 6 different prediction models: taking the previous value forwards (red circle), taking the difference forwards (yellow triangle), LTF (green square), XGboost with default parameters (light blue plus sign), XGboost with tuned hyperparameters (dark blue square with multiplication sign), and XGboost with previous antibiotic usage alone as input features (with default parameters, no information on previous resistance prevalence, pink star). n = 119 Trusts, however not all Trusts contributed data to each boxplot (Supplementary Table 2). Note: 70 residuals that had either missing previous values or previous differences were excluded for comparability of performance measures between the models, although XGboost also made these predictions.

Fig. 4. Predictive performance comparison split by absolute difference between consecutive years.

Mean absolute error for prediction on the test set (resistance prevalence in FY2021–2022) for six different prediction models split by the absolute difference between FY2021–2022 and FY2020–2021 in resistance prevalence, >10% (full circle) or ≤10% (star). Six different prediction models: taking the previous value forwards (red), taking the difference forwards (yellow), LTF (green), XGboost with default parameters (light blue), XGboost with tuned hyperparameters (dark blue), and XGboost with previous antibiotic usage alone as input features (with default parameters, no information on previous resistance prevalence, pink). n Trusts included are provided in the Figure. Note: 70 residuals that had either missing previous values or previous differences were excluded for comparability of performance measures between the models, although XGboost also made these predictions. Results using thresholds of 7.5% and 5% are illustrated in Supplementary Fig. 11. For 3 pathogen–antibiotic combinations: E. coli carbapenems, Klebsiella sp. carbapenems and MSSA vancomycin, most Trusts had 0% resistance prevalence for all available FYs (Supplementary Table 2).