Prevalence dependent calibration of a predictive model for nasal carriage of methicillin-resistant Staphylococcus aureus

Abstract

Background: Published models predicting nasal colonization with Methicillin-resistant Staphylococcus aureus among hospital admissions predominantly focus on separation of carriers from non-carriers and are frequently evaluated using measures of discrimination. In contrast, accurate estimation of carriage probability, which may inform decisions regarding treatment and infection control, is rarely assessed. Furthermore, no published models adjust for MRSA prevalence.

Methods: Using logistic regression, a scoring system (values from 0 to 200) predicting nasal carriage of MRSA was created using a derivation cohort of 3091 individuals admitted to a European tertiary referral center between July 2007 and March 2008. The expected positive predictive value of a rapid diagnostic test (GeneOhm, Becton & Dickinson Co.) was modeled using non-linear regression according to score. Models were validated on a second cohort from the same hospital consisting of 2043 patients admitted between August 2008 and January 2012. Our suggested correction score for prevalence was proportional to the log-transformed odds ratio between cohorts. Calibration before and after correction, i.e. accurate classification into arbitrary strata, was assessed with the Hosmer-Lemeshow-Test.

Results: Treating culture as reference, the rapid diagnostic test had positive predictive values of 64.8% and 54.0% in derivation and internal validation corhorts with prevalences of 2.3% and 1.7%, respectively. In addition to low prevalence, low positive predictive values were due to high proportion (> 66%) of mecA-negative Staphylococcus aureus among false positive results. Age, nursing home residence, admission through the medical emergency department, and ICD-10-GM admission diagnoses starting with "A" or "J" were associated with MRSA carriage and were thus included in the scoring system, which showed good calibration in predicting probability of carriage and the rapid diagnostic test's expected positive predictive value. Calibration for both probability of carriage and expected positive predictive value in the internal validation cohort was improved by applying the correction score.

Conclusions: Given a set of patient parameters, the presented models accurately predict a) probability of nasal carriage of MRSA and b) a rapid diagnostic test's expected positive predictive value. While the former can inform decisions regarding empiric antibiotic treatment and infection control, the latter can influence choice of screening method.

Figure 1

Graphical representation of the linear relationship between age and log-transformed odds of carriage within the derivation cohort according to univariate logistic regression.

Figure 2

Fit of logistic regression model in the prediction of carriage probability in the derivation cohort. Open circles labeled “observed” represent probability within intervals represented by median score. Interval limits were chosen to ensure equal number of individuals per interval.

Figure 3

Fit of non-linear regression model describing the rapid diagnostic test’s positive predictive value depending on the score within the derivation cohort. Full circles labeled “observed” represent positive predictive value within intervals represented by median score. Interval limits were chosen to ensure equal number of positive results per interval.

Figure 4

Nomogram describing functional relationship between score, probability of nasal carriage of Methicillin-resistant Staphylococcus aureus, and the rapid diagnostic test’s positive predictive value.

Figure 5

Nomogram depicting relationship between prevalence and point values to be added to the score as an adjustment for prevalence.

Figure 6

Fit of predictive model in the internal validation cohort with and without correction. Open circles labeled “observed” represent probabilities of carriage within intervals represented by median score. Interval limits were chosen to ensure equal number of individuals per interval.

Figure 7

Fit of model predicting the rapid diagnostic test’s positive predictive value in the validation cohort. Full circles labeled “observed” represent positive predictive values within intervals represented by median score. Interval limits were chosen to ensure equal number of positive results per interval.