Reliability of the American Community Survey Estimates of Risk-Adjusted Readmission Rankings for Hospitals Before and After Peer Group Stratification

Abstract

Importance: Since the transition to the American Community Survey, data uncertainty has complicated its use for policy making and research, despite the ongoing need to identify disparities in health care outcomes. The US Centers for Medicare & Medicaid Services' new, stratified payment adjustment method for its Hospital Readmissions Reduction Program may be able to reduce the reliance on data linkages to socioeconomic survey estimates.

Objective: To determine whether there are differences in the reliability of socioeconomically risk-adjusted hospital readmission rates among hospitals that serve a disproportionate share of low-income populations after stratifying hospitals into peer group-based classification groups.

Design, setting, and participants: This cross-sectional study uses data from the 2014 New York State Health Cost and Utilization Project State Inpatient Database for 96 278 hospital admissions for acute myocardial infarction, pneumonia, and congestive heart failure. The analysis included patients aged 18 years and older who were not transferred to another hospital, who were discharged alive, who did not leave the hospital against medical advice, and who were discharged before December 2014.

Main outcomes and measures: The main outcomes were 30-day hospital readmissions after acute myocardial infarction, pneumonia, and congestive heart failure assessed using hierarchical logistic regression.

Results: The mean (SD) age of the patients was 69.6 (16.0) years for the safety-net hospitals and 74.9 (14.7) years for the non-safety-net hospitals; 9382 (48.8%) and 7003 (48.5%) patients, respectively, were female. For safety net designations, 20% (3 of 15) of all evaluations concealed and distorted differences in risk, with factors such as poverty failing to identify similar risk of acute myocardial infarction readmission until unreliable estimates were excluded from the analysis (OR, 1.23 [95% CI, 1.00-1.52], P = .02; vs OR, 1.17 [95% CI, 0.94-1.46], P = .15). By comparison, 2 of the 60 models (3%) for the peer group-based classification altered the association between socioeconomic status and readmission risk, concealing similarities in congestive heart failure readmission when adjusted using high school completion rates (OR, 1.27 [95% CI 1.02-1.58], P = .04; vs OR, 1.23 [95% CI, 0.98-1.53], P = .06) and distorting similarities in pneumonia readmissions when accounting for the proportion of lone-parent families (OR, 1.27 [95% CI, 0.98-1.66], P = .07; vs OR, 1.35 [95% CI, 1.02-1.80], P = .04) between the lowest and highest socioeconomic status hospitals in quartile 1.

Conclusions and relevance: There was greater precision in socioeconomic adjusted readmission estimates when hospitals were stratified into the new payment adjustment criteria compared with safety net designations. A contributing factor for improved reliability of American Community Survey estimates under the new payment criteria was the merging of patients from low-income neighborhoods with greater homogeneity in survey estimates into groupings similar to those for higher-income patients, whose neighborhoods often exhibit greater estimate variability. Additional efforts are needed to explore the effect of measurement error on American Community Survey-adjusted readmissions using the new peer group-based classification methods.

Figure.. Reliability of American Community Survey (ACS) Socioeconomic Status (SES) Estimates and Cumulative Frequency of Unreliable Estimates

A, Proportions of reliable and unreliable SES estimates owing to sampling errors in the ACS SES estimates released for New York State for the 2014 five-year data cycle are shown by zip code. Reliability was measured using margins of errors and coefficient of variation (CV) with 95% confidence limits, with CV scores of up to 11% and 12% to 40% considered reliable, and scores greater than 40% considered unreliable. B, The cumulative distribution of unreliable SES estimates (CV score >40%) across different hospital grouping quartiles is also shown. SNH indicates safety-net hospital.