Short Term Survival after Admission for Heart Failure in Sweden: Applying Multilevel Analyses of Discriminatory Accuracy to Evaluate Institutional Performance

Abstract

Background: Hospital performance is frequently evaluated by analyzing differences between hospital averages in some quality indicators. The results are often expressed as quality charts of hospital variance (e.g., league tables, funnel plots). However, those analyses seldom consider patients heterogeneity around averages, which is of fundamental relevance for a correct evaluation. Therefore, we apply an innovative methodology based on measures of components of variance and discriminatory accuracy to analyze 30-day mortality after hospital discharge with a diagnosis of Heart Failure (HF) in Sweden.

Methods: We analyzed 36,943 patients aged 45-80 treated in 565 wards at 71 hospitals during 2007-2009. We applied single and multilevel logistic regression analyses to calculate the odds ratios and the area under the receiver-operating characteristic (AUC). We evaluated general hospital and ward effects by quantifying the intra-class correlation coefficient (ICC) and the increment in the AUC obtained by adding random effects in a multilevel regression analysis (MLRA). Finally, the Odds Ratios (ORs) for specific ward and hospital characteristics were interpreted jointly with the proportional change in variance (PCV) and the proportion of ORs in the opposite direction (POOR).

Findings: Overall, the average 30-day mortality was 9%. Using only patient information on age and previous hospitalizations for different diseases we obtained an AUC = 0.727. This value was almost unchanged when adding sex, country of birth as well as hospitals and wards levels. Average mortality was higher in small wards and municipal hospitals but the POOR values were 15% and 16% respectively.

Conclusions: Swedish wards and hospitals in general performed homogeneously well, resulting in a low 30-day mortality rate after HF. In our study, knowledge on a patient's previous hospitalizations was the best predictor of 30-day mortality, and this information did not improve by knowing the sex and country of birth of the patient or where the patient was treated.

Fig 1. Study Population.

Flow diagram showing the selection of patients with first diagnosis of heart failure between 2007 and2009 who were included in the study population.

Fig 2. Ranking of the 71 hospitals (A) and 565 wards (B) according to their 30-day mortality after hospitalization for heart failure (2007–2009).

Values are logarithm odds ratios (i.e., shrunken residuals) with 95% confidence intervals (vertical lines) adjusted for mortality risk score, sex and ethnic origin (see model 3 in Tables 3 and 4). The figure also indicates the values of the hospital and wards intra-class correlation coefficients (ICC) for 30-day mortality.

Fig 3. Receiver operating characteristics (ROC) curves and areas under the ROC curves (AUC) for the different models analyzed in the study.

Model 1 (black line) is a simple logistic regression model including the individual risk score. Model 2 (grey line) is as model 1 but adding sex and ethnicity in categories. Model 3 is as model 2 but adding information on hospitals and wards in a multilevel logistic regression analysis. The ROC curve for model 3 is split showing the contribution of the ward level (thick dotted line) and of the hospital level (thin dotted line).