Assessment of hospital performance with a case-mix standardized mortality model using an existing administrative database in Japan

Abstract

Background: Few studies have examined whether risk adjustment is evenly applicable to hospitals with various characteristics and case-mix. In this study, we applied a generic prediction model to nationwide discharge data from hospitals with various characteristics.

Method: We used standardized data of 1,878,767 discharged patients provided by 469 hospitals from July 1 to October 31, 2006. We generated and validated a case-mix in-hospital mortality prediction model using 50/50 split sample validation. We classified hospitals into two groups based on c-index value (hospitals with c-index > or = 0.8; hospitals with c-index < 0.8) and examined differences in their characteristics.

Results: The model demonstrated excellent discrimination as indicated by the high average c-index and small standard deviation (c-index = 0.88 +/- 0.04). Expected mortality rate of each hospital was highly correlated with observed mortality rate (r = 0.693, p < 0.001). Among the studied hospitals, 446 (95%) had a c-index of >/=0.8 and were classified as the higher c-index group. A significantly higher proportion of hospitals in the lower c-index group were specialized hospitals and hospitals with convalescent wards.

Conclusion: The model fits well to a group of hospitals with a wide variety of acute care events, though model fit is less satisfactory for specialized hospitals and those with convalescent wards. Further sophistication of the generic prediction model would be recommended to obtain optimal indices to region specific conditions.

Figure 1

Predicted and observed mortality by risk decile (n = 939,358). The horizontal axis shows ten predicted mortality ranges. The total number of patients in these ranges is shown in the lower columns. The observed mortality rate with its associated 95% confidence interval is shown by the dark square. The predicted mortality rate is indicated by bar graphs. The c-index of the model is 0.882.

Figure 2

Expected versus observed hospital mortality rate (n = 468*). Each dot represents data from one hospital (r = 0.693, p < 0.001). Expected mortality rates represent average predicted risk in each hospital. *One hospital (mortality rate = 0%) was excluded from this analysis.