Incorporating Acute Conditions into Risk-Adjustment for Provider Profiling: The Case of the US News and World Report Best Hospitals Rankings Methodology

Abstract

Several years ago, the US News and World Report changed their risk-adjustment methodology, now relying almost exclusively on chronic conditions for risk adjustment. The impacts of adding selected acute conditions like pneumonia, sepsis, and electrolyte disorders ("augmented") to their current risk models ("base") for 4 specialties-cardiology, neurology, oncology, and pulmonology-on estimates of hospital performance are reported here. In the augmented models, many acute conditions were associated with substantial risks of mortality. Compared to the base models, the discrimination and calibration of the augmented models for all specialties were improved. While estimated hospital performance was highly correlated between the 2 models, the inclusion of acute conditions in risk-adjustment models meaningfully improved the predictive ability of those models and had noticeable effects on hospital performance estimates. Measures or conditions that address disease severity should always be included when risk-adjusting hospitalization outcomes, especially if the goal is provider profiling.

Figure 1.

Scatterplot of hospital standardized mortality ratios from the base model compared to the augmented model by specialty. Each point represents a hospital, comparing that hospital’s performance estimate from the base model against its performance estimate from the augmented model. Only hospitals with sufficient volume in each specialty are included in the graphs. The number of included hospitals by specialty were: 708 for cardiology; 1243 for neurology; 900 for oncology; and 1697 for pulmonology. The solid lines indicate unchanged relative hospital performance between models. The dotted lines indicate a change in relative hospital performance by 1 standard deviation in either direction. A point below the line indicates a hospital with better relative performance estimated by the augmented model compared to the base model. A point above the line indicates a hospital with worse relative performance estimated by the augmented model compared to the base model.