Personal clinical history predicts antibiotic resistance of urinary tract infections

Abstract

Antibiotic resistance is prevalent among the bacterial pathogens causing urinary tract infections. However, antimicrobial treatment is often prescribed 'empirically', in the absence of antibiotic susceptibility testing, risking mismatched and therefore ineffective treatment. Here, linking a 10-year longitudinal data set of over 700,000 community-acquired urinary tract infections with over 5,000,000 individually resolved records of antibiotic purchases, we identify strong associations of antibiotic resistance with the demographics, records of past urine cultures and history of drug purchases of the patients. When combined together, these associations allow for machine-learning-based personalized drug-specific predictions of antibiotic resistance, thereby enabling drug-prescribing algorithms that match an antibiotic treatment recommendation to the expected resistance of each sample. Applying these algorithms retrospectively, over a 1-year test period, we find that they greatly reduce the risk of mismatched treatment compared with the current standard of care. The clinical application of such algorithms may help improve the effectiveness of antimicrobial treatments.

Extended Data Figure 1:. Availability of resistance measurements over time.

For each of the 6 antibiotics, the fraction of urine samples for which resistance was measured, overall (black) and for each of the three most common species (colors), is plotted across the 10-year sampling period. Also indicated are the time ranges used for model Training (green horizontal bars) and Testing (red bars). Time periods during which measurements of resistance to cephalexin were scarce were removed from analysis (gray bar).

Extended Data Figure 2:. Frequency of resistance over time.

Frequencies of resistance for each of the three common species (colored lines) and the overall sample (black lines) over the 10 year dataset. Empty time intervals correspond to periods during which resistance was not frequently measured (matching the gray horizontal bar of Extended Data Fig. 1).

Extended Data Figure 3:. Odds of resistance as a function of age for different demographic groups.

Frequency of resistance to each of the 6 antibiotics, in each of 10 age bins (0,10,…,100 years). (a) Frequencies of resistance for five non-overlapping demographic groups: men not residing in retirement homes (blue), men residing in retirement homes (dotted blue), women not pregnant and not residing in retirement homes (magenta), women in retirement homes (magenta dotted), and pregnant women (red). (b) Comparing the overall frequency of resistance to the 6 drugs for women and men across age.

Extended Data Figure 4:. Odds ratios of resistance to each of the antibiotics for past purchases of different drugs across a range of purchase-to-sample time intervals: adjustments for demographics and cross-resistance.

(a) Multivariate logistic regression models for the association of each antibiotic resistance with past purchases of the indicated drugs not accounting for cross-resistance (Online Methods: Logistic regression “Purchase history”. Same graphical scheme as in Fig. 4a,b). (b) Logistic regression model as in (a) adjusted for cross-resistance (Online Methods: Logistic regression “Purchase history adjusted for cross resistance”). (c) Logistic regression model as in (a) adjusted for demographics (Online Methods: Logistic regression “Purchase history adjusted for demographics”. Gray asterisks indicate statistical significance and non-significant values, with Bonferroni corrected P>0.05, are blanked.

Extended Data Figure 5:. Correlations among resistances to different antibiotics.

Correlation among resistance measurements for each pair of antibiotics across all samples for which both resistances were measured. Cephalexin and cefuroxime axetil, which have a particularly high correlation (marked with ‘x’), are treated as “analogous” in the analysis of indirect effects of purchases on resistance (Online Methods: Logistic regression “Purchase history adjusted for cross-resistance”).

Extended Data Figure 6:. Model performance on test and training data. Area Under Curve (AUC) for Receiver Operator Characteristic for prediction of resistance based on demographics, sample history and purchase history, individually and in a complete model combining all feature sets.

Each feature set was modelled using Logistic Regression (LR), and the complete model was modelled by both LR and Gradient Boosting Decision Trees (GBDT). To identify overfitting, model performance on the testing dataset (grey) was contrasted with model performance on the training dataset (black; Supplementary Fig. 2 for definition of training and test time periods). Mild level of overfitting is seen for all drugs except trimethoprim which showed no over fitting.

Extended Data Figure 7:. The fraction of samples that can be treated by at least one drug given set thresholds on the single-drug resistance probability scores.

Given the complete-model assigned probabilities of resistance $P_k^m$ of each sample m to each antibiotic k, we calculated the fraction of samples, within the one-year test period, that have at least one drug with resistance score below a threshold. This fraction is calculated assuming that the threshold used to determine resistance of single drugs is either: (a) the same probability threshold P^threshold for all drugs (counting all samples for which $P_k^m<P^{\mathit{\text{threshold}}}$ for at least one antibiotics k), or (b) the same rank threshold r^threshold for all drugs, counting all samples for which $P_k^m<P_k^{\mathit{\text{threshold}}}\left(r^{\mathit{\text{threshold}}}\right)$ for at least one antibiotics k, where $P_k^{\mathit{\text{threshold}}}\left(r^{\mathit{\text{threshold}}}\right)$ is the probability threshold of drug k that include a fraction r^threshold of the samples.

Extended Data Figure 8:. Schematic diagram of ML-trained prescription models.

A set of samples with features of demographics, sample resistance history and antibiotic purchase history labelled for resistance to each antibiotic k (‘Train set’) is used to train an antibiotic resistance prediction model (Online Methods: Logistic regression, terms #1–#9). The model is applied to an SDET set of cases from the test period to calculate probabilities of resistance to each antibiotic. In an unconstrained model the antibiotic with minimal probability for resistance is suggested. The calculated probabilities of resistance together with the respective prescriptions of the SDET set of cases are used to add a “cost” term. In a constrained drug prescription model, the antibiotic with the minimal cost-adjusted probability is suggested.

Extended Data Figure 9:. Robustness of ML-trained prescription models across age and gender and with respect to the clinical definition of resistance.

(a) Frequency of mismatched treatment across all SDET cases, comparing physician’s prescriptions (dark bar) to algorithmic recommendations by the constrained and unconstrained models (cyan and magenta hatched, respectively) for females (top) and males (bottom) separated into 3 major age groups. (b) Frequency of mismatched treatment across all SDET cases (Online Methods), when classifying “Intermediate” level of resistance as “Resistant”. Comparing mismatch frequencies of physicians’ prescriptions (dark bar) to algorithmic recommendations (light bars), either unconstrained (magenta hatched) or constrained for recommending drugs at the same ratio as physicians (cyan hatched). Also presented are the null expectations for randomly prescribing drugs with equal probabilities (Random “Dice”, magenta dashed) or for random drug permutations (Random permutations, cyan dashed).

Figure 1:. Frequency of bacterial species and antibiotic resistance in urinary tract infections.

(a) Species abundance across the entire UTI dataset (July 2007-June 2017, 711099 samples). (b) The frequency of resistance and intermediate resistance to the 6 focal antibiotic drugs for the three most common bacterial species and for the urine sample as a whole (“sample”, defined as the highest resistance measured for each isolate in the sample). Dark to light shades represent resistant, intermediate and sensitive, respectively. (c) Frequencies of resistance for each of the three common species (colored lines) and the sample resistance (black lines) over the 10 year sampling time, for two representative antibiotics: trimethoprim-sulfa (top) and ciprofloxacin (bottom; see Extended Data Fig. 2 for all antibiotics). Data points represent quarterly averages.

Figure 2:. Antibiotic-specific associations of resistance with demographic factors.

(a) Distribution of urine cultures across major demographic factors: age, gender (top, females; bottom, males), pregnancy (red) and retirement home residence (dark). (b) Adjusted odds ratios of resistance for each demographic variable (see Logistic regression – demographics in the Online Methods, and see Supplementary Table 2 for all adjusted and unadjusted regression coefficients). Asterisks indicate statistical significance and non-significant odds ratios (P>0.01) are shown as blank. (c) Frequency of resistance as a function of age showing qualitatively distinct patterns for three representative antibiotics. UTI samples are separated into five non-overlapping categories: men not residing in retirement homes (blue), men residing in retirement homes (dotted blue), women not pregnant and not residing in retirement homes (magenta), women in retirement homes (magenta dotted), and pregnant women (red). See Extended Data Fig. 3 for all antibiotics.

Figure 3:. Long term “memory” of resistance across same-patient samples.

(a,b) Risk ratio of the resistance of a urine sample given a record of a resistant versus sensitive earlier sample from the same patient, as a function of the time difference between the two samples, for trimethoprim-sulfa (a) and ciprofloxacin (b, See Online Methods and Supplementary Fig. 1 for all antibiotics). Risk ratios are well fitted with ${\zeta }_{\mathit{\text{pairs}}}\simeq C_m{e}^{t/{\tau }_m}+C_{\infty }$, representing a time-decaying correlation (“memory”, yellow) and a time-independent correlation (“patient propensity”, green) among sample pairs. The magnitudes of these terms are shown as stacked bars on the right and the memory time (τ_m) is indicated across the time axis (yellow arrow). Gray triangle and diamond represent trimethoprim-sulfa and ciprofloxacin respectively, linking between the different panels. (c) Time scale of the memory of resistance τ_m for the 6 different antibiotics (correlated with the yellow arrows in panels (a) and (b). (d) The magnitude of long-term and timeless memory for the different antibiotics (yellow, green bars, respectively).

Figure 4:. Direct association of past purchase with its cognate resistance leads, through association among resistances, to indirect association of purchases with noncognate resistances.

(a) Multivariate logistic regression models for the association of resistance to trimethoprim-sulfa (left) and ciprofloxacin (right) with past purchases of the indicated drugs at the indicated time intervals prior to infection (“Total”, See Extended Data Fig. 4a for all antibiotics; Logistic regression - purchase history in Online Methods). Values represent the odds ratios for a single purchase of a specific drug at a specific time interval (color map, stars for statistical significance as indicated, non-significant values, with Bonferroni corrected P>0.05, are blanked). A long term association is observed between resistance and past purchase of its matching (cognate, arrows) as well as with non-cognate antibiotics. (b) Logistic regression model as in (a) adjusted for cross-resistance. This adjusted model diminishes or even completely removes noncognate drug-to-resistance associations while fully preserving the cognate associations (“Direct”, See Extended Data Fig. 4b for all antibiotics; arrows; cyan, trimethoprim-sulfa; magenta, ciprofloxacin). (c,d) Association of resistance to trimethoprim-sulfa (c) and ciprofloxacin (d) with purchases of these two drug (cyan and magenta, respectively). Note differences between total (dashed lines) and direct (solid lines) effects for cognate (thick lines) versus noncognate (thin lines) drugs.

Figure 5:. Algorithmically suggesting antibiotic prescription for empirical treatments can much improve upon the current standard-of-care.

(a) For each of the 6 antibiotics, we calculated the fraction (top) of resistant (red) and sensitive (green) samples, as well as the risk of resistance (bottom), for all samples within the one-year test period whose complete-model machine-learning assigned probabilities of resistance $P_k^m$ were below a set threshold P^threshold (x-axis, see Supplementary Fig. 2 for all antibiotics and more formal definitions). At P^threshold = 1 the risk of sample resistance equals the population-wide risk of resistance (dotted red line). Setting P^threshold=0.12 would permit treatment of 75% of these infections with much reduced risk of resistance compared to population-wide risk (48% reduction, down-pointing arrow). (b) Differentiation between samples resistant to cefuroxime axetil and sensitive to nitrofurantoin (red) and vice versa (blue) by their model-assigned resistance probabilities (odds ratio of 3.9 for red points below the diagonal and blue points above it; P<10⁻¹⁰⁰, Fisher exact; see Supplementary Fig. 3 for all pairs of antibiotics). (c) Physician’s frequency of mismatched prescriptions across all SDET cases (dark bar) was slightly better than null expectation for randomly prescribing drugs with equal probabilities (Random “dice”, magenta dashed, P<10⁻¹⁰) or for randomly permuting the physicians’ prescriptions (Random permutations, cyan dashed, P=2.5×10⁻⁵). These mismatch treatment rates were substantially reduced by the machine-learning (ML) based recommendations (light bars,), either unconstrained (magenta hatched, P<10⁻¹⁰) or constrained to recommend drugs at the exact same frequencies prescribed by the physicians (cyan hatched, P<10⁻¹⁰). (d) Top, distribution of the drugs prescribed by the physicians (dark bar), by the constrained algorithm (cyan-hashed light bar, constrained to be equal to the Physician’s) and by the unconstrained algorithm (magenta-hashed light bar). Bottom, for each of these prescription models, the frequency of mismatched treatment for each of the drugs is indicated, normalized by the expected mismatch frequency for random drug prescription (the average rate of resistance to the drug across the SDET population).