Comparing bloodstream infection rates: the effect of indicator specifications in the evaluation of processes and indicators in infection control (EPIC) study

Abstract

Objective: Bloodstream infection (BSI) rates are used as comparative clinical performance indicators; however, variations in definitions and data-collection approaches make it difficult to compare and interpret rates. To determine the extent to which variation in indicator specifications affected infection rates and hospital performance rankings, we compared absolute rates and relative rankings of hospitals across 5 BSI indicators.

Design: Multicenter observational study. BSI rate specifications varied by data source (clinical data, administrative data, or both), scope (hospital wide or intensive care unit specific), and inclusion/exclusion criteria. As appropriate, hospital-specific infection rates and rankings were calculated by processing data from each site according to 2-5 different specifications.

Setting: A total of 28 hospitals participating in the EPIC study.

Participants: Hospitals submitted deidentified information about all patients with BSIs from January through September 1999.

Results: Median BSI rates for 2 indicators based on intensive care unit surveillance data ranged from 2.23 to 2.91 BSIs per 1000 central-line days. In contrast, median rates for indicators based on administrative data varied from 0.046 to 7.03 BSIs per 100 patients. Hospital-specific rates and rankings varied substantially as different specifications were applied; the rates of 8 of 10 hospitals were both greater than and less than the mean. Correlations of hospital rankings among indicator pairs were generally low (rs=0-0.45), except when both indicators were based on intensive care unit surveillance (rs = 0.83).

Conclusions: Although BSI rates seem to be a logical indicator of clinical performance, the use of various indicator specifications can produce remarkably different judgments of absolute and relative performance for a given hospital. Recent national initiatives continue to mix methods for specifying BSI rates; this practice is likely to limit the usefulness of such information for comparing and improving performance.