Development and Validation of an Agency for Healthcare Research and Quality Indicator for Mortality After Congenital Heart Surgery Harmonized With Risk Adjustment for Congenital Heart Surgery (RACHS-1) Methodology

Abstract

Background: The National Quality Forum previously approved a quality indicator for mortality after congenital heart surgery developed by the Agency for Healthcare Research and Quality (AHRQ). Several parameters of the validated Risk Adjustment for Congenital Heart Surgery (RACHS-1) method were included, but others differed. As part of the National Quality Forum endorsement maintenance process, developers were asked to harmonize the 2 methodologies.

Methods and results: Parameters that were identical between the 2 methods were retained. AHRQ's Healthcare Cost and Utilization Project State Inpatient Databases (SID) 2008 were used to select optimal parameters where differences existed, with a goal to maximize model performance and face validity. Inclusion criteria were not changed and included all discharges for patients <18 years with International Classification of Diseases, Ninth Revision, Clinical Modification procedure codes for congenital heart surgery or nonspecific heart surgery combined with congenital heart disease diagnosis codes. The final model includes procedure risk group, age (0-28 days, 29-90 days, 91-364 days, 1-17 years), low birth weight (500-2499 g), other congenital anomalies (Clinical Classifications Software 217, except for 758.xx), multiple procedures, and transfer-in status. Among 17 945 eligible cases in the SID 2008, the c statistic for model performance was 0.82. In the SID 2013 validation data set, the c statistic was 0.82. Risk-adjusted mortality rates by center ranged from 0.9% to 4.1% (5th-95th percentile).

Conclusions: Congenital heart surgery programs can now obtain national benchmarking reports by applying AHRQ Quality Indicator software to hospital administrative data, based on the harmonized RACHS-1 method, with high discrimination and face validity.

Keywords: congenital heart defects; mortality; pediatrics; risk factors.

Figure 1

A, Calibration of harmonized model based on Hosmer–Lemeshow goodness‐of‐fit test (Healthcare Cost and Utilization Project [HCUP] State Inpatient Databases [SID] 2008). B, Calibration of harmonized model based on Hosmer–Lemeshow goodness‐of‐fit test (HCUP SID 2013). Circles represent observed and expected mortality rates within each decile of risk. The solid line represents the linear regression of observed in‐hospital mortality rate vs expected in‐hospital mortality rate. The dashed line represents the situation where observed and expected rates are identical. Source: AHRQ.24, 25

Figure 2

A, Calibration of the Agency for Healthcare Research and Quality (AHRQ) Pediatric Quality Indicator (PDI) 06 model based on Hosmer–Lemeshow goodness‐of‐fit test (Healthcare Cost and Utilization Project [HCUP] State Inpatient Databases [SID] 2013). B, Calibration of Risk Adjustment for Congenital Heart Surgery (RACHS‐1) model based on Hosmer–Lemeshow goodness‐of‐fit test (HCUP SID 2013). Circles represent observed and expected mortality rates within each decile of risk. The solid line represents the linear regression of observed in‐hospital mortality rate vs expected in‐hospital mortality rate. The dashed line represents the situation where observed and expected rates are identical. Source: AHRQ.24, 25