Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Mar 19;4(1):e0044.
doi: 10.2106/JBJS.OA.18.00044. eCollection 2019 Mar 27.

When Stars Do Not Align: Overall Hospital Quality Star Ratings and the Volume-Outcome Association

Mark Alan Fontana  1   2 , Stephen Lyman  1   2 , Wasif Islam  1   2 , Catherine H MacLean  1
Affiliations

When Stars Do Not Align: Overall Hospital Quality Star Ratings and the Volume-Outcome Association

Mark Alan Fontana et al. JB JS Open Access. .

Abstract

Background: Volume-outcome relationships are well established for coronary artery bypass grafting and total joint arthroplasty surgery. Although the U.S. Centers for Medicare & Medicaid Services (CMS) Overall Hospital Quality Star Ratings program includes outcome quality measures for these procedures, these outcome quality measures are not counted toward the star ratings for low-volume hospitals. We sought to assess whether excluding low-volume hospitals from surgical quality measures with known volume-outcome relationships affects the star ratings.

Methods: We identified quality measures used in CMS's star ratings that are related to surgical procedures with a known volume-outcome relationship and tested for the presence of the volume-outcome association for each of these measures. We then imputed missing values for low-volume hospitals for each measure and otherwise identically repeated the CMS calculations in order to assess the percentages of hospitals with the same, better, or worse ratings.

Results: Among the measures used to calculate star ratings, we identified 4 quality measures (2 related to coronary artery bypass grafting and 2 related to total joint arthroplasty) with known volume-outcome relationships that were excluded from the calculations of the star ratings for low-volume hospitals. We confirmed a volume-outcome association in the CMS data for all 4 measures. When total joint arthroplasty complications were imputed for low-volume hospitals and then included in the calculation of the star ratings, over one-third of hospitals received a different rating; both low-volume and other hospitals were more often hurt than helped. Imputing the other 3 quality measures among low-volume hospitals left the ratings unchanged.

Conclusions: The CMS star ratings do not fully represent the risks of undergoing procedures at low-volume hospitals, potentially misrepresent quality across facilities, and hence are of uncertain utility to consumers.

PubMed Disclaimer

Figures

Fig. 1
Fig. 1
Star comparison matrices: replication versus imputation. Entries highlighted in blue indicate the number of hospitals that have identical star ratings between replication (without imputation) and imputation with use of the regression-based imputation method. Other entries not highlighted indicate the number of hospitals that have different star ratings between replication and imputation. Comparison and total columns indicate the number (and percentage) of hospitals with the same, better, or worse star ratings after imputation. “Newly rated” indicates the number of hospitals that did not have a star rating before imputation but did after imputation. “Never rated” indicates the number of hospitals that did not have a star rating before or after imputation.
Fig. 2
Fig. 2
Composite hospital scores: replication versus imputation. The green triangles represent hospitals for which total joint arthroplasty complications were imputed with use of the regression-based imputation method, and the blue diamonds represent hospitals without imputed total joint arthroplasty complications.
Fig. 3
Fig. 3
Safety domain hospital scores: replication versus imputation. The green triangles represent hospitals for which total joint arthroplasty complications were imputed with use of the regression-based imputation method, and the blue diamonds represent hospitals without imputed total joint arthroplasty complications.
See this image and copyright information in PMC

References

    1. Hartman M, Martin AB, Espinosa N, Catlin A; The National Health Expenditure Accounts Team. National health care spending in 2016: spending and enrollment growth slow after initial coverage expansions. Health Aff (Millwood). 2018. January;37(1):150-60. Epub 2017 Dec 6. - PubMed
    1. Centers for Medicare & Medicaid Services. National health expenditures fact sheet. 2018. December 6 https://www.cms.gov/research-statistics-data-and-systems/statistics-tren.... Accessed 2018 Dec 14.
    1. Centers for Medicare & Medicaid Services. CMS updates website to compare hospital quality. 2017. December https://www.cms.gov/newsroom/press-releases/cms-updates-website-compare-.... Accessed 2019 Feb 14.
    1. Venkatesh AK, Bernheim SM, Qin L, Bao H, Simoes J, Norton E, Wing M, Glennon G, Shah R, Herrin J, Lin H, Lin Z, Krumholz HM. Overall hospital quality star rating on Hospital Compare December 2017 updates and specifications report. 2017. December https://s3.amazonaws.com/assets.fiercemarkets.net/public/004-Healthcare/.... Accessed 2018 Dec 14.
    1. Venkatesh AK, Bernheim SM, Qin L, Bao H, Simoes J, Norton E, Wing M, Glennon G, Shah R, Herrin J, Lin H, Lin Z, Krumholz HM. Overall hospital quality star ratings on Hospital Compare, methodology report (v2.0). 2016. May https://cpb-us-w2.wpmucdn.com/u.osu.edu/dist/c/28860/files/2016/08/Star_.... Accessed 2018 Dec 14.