Can Machine Learning Help Identify Patients at Risk for Recurrent Sexually Transmitted Infections?

Abstract

Background: A substantial fraction of sexually transmitted infections (STIs) occur in patients who have previously been treated for an STI. We assessed whether routine electronic health record (EHR) data can predict which patients presenting with an incident STI are at greatest risk for additional STIs in the next 1 to 2 years.

Methods: We used structured EHR data on patients 15 years or older who acquired an incident STI diagnosis in 2008 to 2015 in eastern Massachusetts. We applied machine learning algorithms to model risk of acquiring ≥1 or ≥2 additional STIs diagnoses within 365 or 730 days after the initial diagnosis using more than 180 different EHR variables. We performed sensitivity analysis incorporating state health department surveillance data to assess whether improving the accuracy of identifying STI cases improved algorithm performance.

Results: We identified 8723 incident episodes of laboratory-confirmed gonorrhea, chlamydia, or syphilis. Bayesian Additive Regression Trees, the best-performing algorithm of any single method, had a cross-validated area under the receiver operating curve of 0.75. Receiver operating curves for this algorithm showed a poor balance between sensitivity and positive predictive value (PPV). A predictive probability threshold with a sensitivity of 91.5% had a corresponding PPV of 3.9%. A higher threshold with a PPV of 29.5% had a sensitivity of 11.7%. Attempting to improve the classification of patients with and without repeat STIs diagnoses by incorporating health department surveillance data had minimal impact on cross-validated area under the receiver operating curve.

Conclusions: Machine algorithms using structured EHR data did not differentiate well between patients with and without repeat STIs diagnosis. Alternative strategies, able to account for sociobehavioral characteristics, could be explored.

Figure 1.

BART* regression model distribution of predicted risk scores, ≥2 repeat diagnoses of sexually transmitted infections (STIs)^† within 730 days, Atrius Health, 2008 to 2015. A, Risk score distribution for patients with exactly 2 repeat STIs. B, Risk score distribution for patients with exactly 3 repeat STIs. C, Risk score distribution for patients with exactly 4 repeat STIs. D, Risk score distribution for patients with ≥5 repeat STIs. *BART indicates Bayesian Additive Regression Trees. ^†Defined as positive laboratory result for chlamydia, gonorrhea, or a syphilis diagnosis.