When policy gets it right: variability in u.s. Hospitals' diagnosis of ventilator-associated pneumonia*

Abstract

Objective: The Centers for Disease Control has recently proposed a major change in how ventilator-associated pneumonia is defined. This has profound implications for public reporting, reimbursement, and accountability measures for ICUs. We sought to provide evidence for or against this change by quantifying limitations of the national definition of ventilator-associated pneumonia that was in place until January 2013, particularly with regard to comparisons between, and ranking of, hospitals and ICUs.

Design: A prospective survey of a nationally representative group of 43 hospitals, randomly selected from the American Hospital Association Guide (2009). Subjects classified six standardized vignettes of possible cases of ventilator-associated pneumonia as pneumonia or no pneumonia.

Subjects: Individuals responsible for ventilator-associated pneumonia surveillance at 43 U.S. hospitals.

Interventions: None.

Measurements and main results: We measured the proportion of standardized cases classified as ventilator-associated pneumonia. Of 138 hospitals consented, 61 partially completed the survey and 43 fully completed the survey (response rate 44% and 31%, respectively). Agreement among hospitals about classification of cases as ventilator-associated pneumonia/not ventilator-associated pneumonia was nearly random (Fleiss κ 0.13). Some hospitals rated 0% of cases as having pneumonia; others classified 100% as having pneumonia (median, 50%; interquartile range, 33-66%). Although region of the country did not predict case assignment, respondents who described their region as "rural" were more likely to judge a case to be pneumonia than respondents elsewhere (relative risk, 1.25, Kruskal-Wallis chi-square, p = 0.03).

Conclusions: In this nationally representative study of hospitals, assignment of ventilator-associated pneumonia is extremely variable, enough to render comparisons between hospitals worthless, even when standardized cases eliminate variability in clinical data abstraction. The magnitude of this variability highlights the limitations of using poorly performing surveillance definitions as methods of hospital evaluation and comparison, and our study provides very strong support for moving to a more objective definition of ventilator-associated complications.