Quality of traditional surveillance for public reporting of nosocomial bloodstream infection rates

Abstract

Context: Central line-associated bloodstream infection (BSI) rates, determined by infection preventionists using the Centers for Disease Control and Prevention (CDC) surveillance definitions, are increasingly published to compare the quality of patient care delivered by hospitals. However, such comparisons are valid only if surveillance is performed consistently across institutions.

Objective: To assess institutional variation in performance of traditional central line-associated BSI surveillance.

Design, setting, and participants: We performed a retrospective cohort study of 20 intensive care units among 4 medical centers (2004-2007). Unit-specific central line-associated BSI rates were calculated for 12-month periods. Infection preventionists, blinded to study participation, performed routine prospective surveillance using CDC definitions. A computer algorithm reference standard was applied retrospectively using criteria that adapted the same CDC surveillance definitions.

Main outcome measures: Correlation of central line-associated BSI rates as determined by infection preventionist vs the computer algorithm reference standard. Variation in performance was assessed by testing for institution-dependent heterogeneity in a linear regression model.

Results: Forty-one unit-periods among 20 intensive care units were analyzed, representing 241,518 patient-days and 165,963 central line-days. The median infection preventionist and computer algorithm central line-associated BSI rates were 3.3 (interquartile range [IQR], 2.0-4.5) and 9.0 (IQR, 6.3-11.3) infections per 1000 central line-days, respectively. Overall correlation between computer algorithm and infection preventionist rates was weak (ρ = 0.34), and when stratified by medical center, point estimates for institution-specific correlations ranged widely: medical center A: 0.83; 95% confidence interval (CI), 0.05 to 0.98; P = .04; medical center B: 0.76; 95% CI, 0.32 to 0.93; P = .003; medical center C: 0.50, 95% CI, -0.11 to 0.83; P = .10; and medical center D: 0.10; 95% CI -0.53 to 0.66; P = .77. Regression modeling demonstrated significant heterogeneity among medical centers in the relationship between computer algorithm and expected infection preventionist rates (P < .001). The medical center that had the lowest rate by traditional surveillance (2.4 infections per 1000 central line-days) had the highest rate by computer algorithm (12.6 infections per 1000 central line-days).

Conclusions: Institutional variability of infection preventionist rates relative to a computer algorithm reference standard suggests that there is significant variation in the application of standard central line-associated BSI surveillance definitions across medical centers. Variation in central line-associated BSI surveillance practice may complicate interinstitutional comparisons of publicly reported central line-associated BSI rates.

Figure 1.. Schematic of Computer Algorithm for Central Line–Associated Bloodstream Infection Surveillance

^aCommon skin commensals are defined as diphtheroids, Bacillus, Propionibacterium, coagulase-negative Staphylococcus, or Micrococcus species. ^bCalendar date of admission to the hospital is considered hospital day 1. ^cActive surveillance screening cultures and catheter tip cultures were not considered to avoid misclassifying episodes as secondary. ^dCentral line presence was assessed on the first day of a primary bloodstream infection episode and through 2 hospital days prior.

Figure 2.. Linear Regression of Rates of Central Line–Associated Bloodstream Infection

The line in each panel represents fitted values.

Figure 3.. Predicted Infection Preventionist Central Line–Associated Bloodstream Infection Rates

The dotted line that transects the graph from 0 represents a hypothetical line of perfect agreement (slope=1). Regression line equations that assign a common slope and varying intercepts: medical center A, y=1.47 + 0.43x; B, y=−0.28 + 0.43x; C, y = −2.99 + 0.43x; D, y=0.35 + 0.43x.

Figure 4.. Relative Ranking of 4 Medical Centers

Within each surveillance group, medical centers are arranged in ascending order by central line–associated bloodstream infection rate. Error bars indicate 95% confidence intervals. Rates were calculated within each medical center by aggregating all intensive care unit rates. With infection preventionist surveillance, the following rates were determined: C, 2.4 (95% CI, 1.8-3.3); D, 3.2 (95% CI, 2.9-3.7); B, 3.3 (95% CI, 2.8-3.8); A, 3.5 (95% CI, 2.7-4.4). With the computer algorithm the following infection rates were determined: A, 4.7 (95% CI, 3.8-5.7); D, 6.7 (95% CI, 6.2-7.3); B, 8.2 (95% CI, 7.4-9.0); C, 12.6 (95% CI, 11.0-14.3).