The ANZROD model: better benchmarking of ICU outcomes and detection of outliers

Abstract

Objective: To compare the impact of the 2013 Australian and New Zealand Risk of Death (ANZROD) model and the 2002 Acute Physiology and Chronic Health Evaluation (APACHE) III-j model as risk-adjustment tools for benchmarking performance and detecting outliers in Australian and New Zealand intensive care units.

Methods: Data were extracted from the Australian and New Zealand Intensive Care Society Adult Patient Database for all ICUs that contributed data between 1 January 2010 and 31 December 2013. Annual standardised mortality ratios (SMRs) were calculated for ICUs using the ANZROD and APACHE III-j models. They were plotted on funnel plots separately for each hospital type, with ICUs above the upper 99.8% control limit considered as potential outliers with worse performance than their peer group. Overdispersion parameters were estimated for both models. Overall fit was assessed using the Akaike information criterion (AIC) and Bayesian information criterion (BIC). Outlier association with mortality was assessed using a logistic regression model.

Results: The ANZROD model identified more outliers than the APACHE III-j model during the study period. The numbers of outliers in rural, metropolitan, tertiary and private hospitals identified by the ANZROD model were 3, 2, 6 and 6, respectively; and those identified by the APACHE III-j model were 2, 0, 1 and 1, respectively. The degree of overdispersion was less for the ANZROD model compared with the APACHE III-j model in each year. The ANZROD model showed better overall fit to the data, with smaller AIC and BIC values than the APACHE III-j model. Outlier ICUs identified using the ANZROD model were more strongly associated with increased mortality.

Conclusion: The ANZROD model reduces variability in SMRs due to casemix, as measured by overdispersion, and facilitates more consistent identification of true outlier ICUs, compared with the APACHE III-j model.