Mortality Measures to Profile Hospital Performance for Patients With Septic Shock

Abstract

Objectives: Sepsis care is becoming a more common target for hospital performance measurement, but few studies have evaluated the acceptability of sepsis or septic shock mortality as a potential performance measure. In the absence of a gold standard to identify septic shock in claims data, we assessed agreement and stability of hospital mortality performance under different case definitions.

Design: Retrospective cohort study.

Setting: U.S. acute care hospitals.

Patients: Hospitalized with septic shock at admission, identified by either implicit diagnosis criteria (charges for antibiotics, cultures, and vasopressors) or by explicit International Classification of Diseases, 9th revision, codes.

Interventions: None.

Measurements and main results: We used hierarchical logistic regression models to determine hospital risk-standardized mortality rates and hospital performance outliers. We assessed agreement in hospital mortality rankings when septic shock cases were identified by either explicit International Classification of Diseases, 9th revision, codes or implicit diagnosis criteria. Kappa statistics and intraclass correlation coefficients were used to assess agreement in hospital risk-standardized mortality and hospital outlier status, respectively. Fifty-six thousand six-hundred seventy-three patients in 308 hospitals fulfilled at least one case definition for septic shock, whereas 19,136 (33.8%) met both the explicit International Classification of Diseases, 9th revision, and implicit septic shock definition. Hospitals varied widely in risk-standardized septic shock mortality (interquartile range of implicit diagnosis mortality: 25.4-33.5%; International Classification of Diseases, 9th revision, diagnosis: 30.2-38.0%). The median absolute difference in hospital ranking between septic shock cohorts defined by International Classification of Diseases, 9th revision, versus implicit criteria was 37 places (interquartile range, 16-70), with an intraclass correlation coefficient of 0.72, p value of less than 0.001; agreement between case definitions for identification of outlier hospitals was moderate (kappa, 0.44 [95% CI, 0.30-0.58]).

Conclusions: Risk-standardized septic shock mortality rates varied considerably between hospitals, suggesting that septic shock is an important performance target. However, efforts to profile hospital performance were sensitive to septic shock case definitions, suggesting that septic shock mortality is not currently ready for widespread use as a hospital quality measure.

Figure 1

Flow chart of primary cohort selection.

Figure 2

Caterpillar plot of risk standardized hospital mortality rates for cohort of septic shock defined by charges for antibiotics/cultures/vasopressors (implicit diagnosis). The horizontal line at 29% is the average mortality rate for the cohort. Points along the caterpillar plot colored green signify low mortality outlier hospitals and red points high mortality outliers. Tick marks along the x-axis demonstrate the hospitals identified as low mortality (green) and high mortality (red) outliers when patient cohorts were defined with ICD-9 criteria for septic shock.

Figure 3

Scatterplot of hospital risk-standardized mortality rates for septic shock defined by antibiotics/cultures/vasopressors as compared with ICD-9 codes. Intra-class correlation coefficient was 0.72.