Development of an algorithm to aid triage decisions for intensive care unit admission: a clinical vignette and retrospective cohort study

Abstract

Background: Intensive care unit (ICU) admission triage is performed routinely and is often based solely on clinical judgment, which could mask biases. A computerized algorithm to aid ICU triage decisions was developed to classify patients into the Society of Critical Care Medicine's prioritization system. In this study, we sought to evaluate the reliability and validity of this algorithm.

Methods: Nine senior physicians evaluated forty clinical vignettes based on real patients. The reference standard was defined as the priorities ascribed by two investigators with full access to patients' records. Agreement of algorithm-based priorities with the reference standard and with intuitive priorities provided by the physicians were evaluated. Correlations between algorithm prioritization and physicians' judgment of the appropriateness of ICU admissions in scarcity and nonscarcity settings were also evaluated. Validity was further assessed by retrospectively applying this algorithm to 603 patients with requests for ICU admission for association with clinical outcomes.

Results: Agreement between algorithm-based priorities and the reference standard was substantial, with a median κ of 0.72 (interquartile range [IQR] 0.52-0.77). Algorithm-based priorities demonstrated higher interrater reliability (overall κ 0.61, 95% confidence interval [CI] 0.57-0.65; median percentage agreement 0.64, IQR 0.59-0.70) than physicians' intuitive prioritization (overall κ 0.51, 95% CI 0.47-0.55; median percentage agreement 0.49, IQR 0.44-0.56) (p = 0.001). Algorithm-based priorities were also associated with physicians' judgment of appropriateness of ICU admission (priorities 1, 2, 3, and 4 vignettes would be admitted to the last ICU bed in 83.7%, 61.2%, 45.2%, and 16.8% of the scenarios, respectively; p < 0.001) and with actual ICU admission, palliative care consultation, and hospital mortality in the retrospective cohort.

Conclusions: This ICU admission triage algorithm demonstrated good reliability and validity. However, more studies are needed to evaluate a difference in benefit of ICU admission justifying the admission of one priority stratum over the others.

Fig. 1

Proportion of scenarios in which patients would be admitted to the intensive care unit (ICU), according to algorithm-based priorities, stratified by ICU scarcity or nonscarcity setting. *p < 0.001; ^† p < 0.001

Fig. 2

Association of priority categories with hospital mortality, intensive care unit (ICU) admission and palliative care consultation. Priorities 1, 2, 3, and 4 were statistically associated with hospital mortality (p < 0.001 by χ² for trend), ICU admission (p = 0.035 by χ² for trend), and palliative care consultation (p = 0.036 by χ² for trend)

Fig. 3

Association of individual questions with hospital mortality (a) and intensive care unit (ICU) admission (b). See supplementary material in Additional file 1 for number of patients in each group. *p < 0.001; ^† p < 0.05. Question 1: 1 = active intervention, 2 = monitoring. Question 2: 1 = no comorbidities, 2 = compensated comorbidities, 3 = decompensated comorbidities, 4 = advanced disease. Question 3: 1 = functionally independent, 2 = partially dependent, 3 = severely dependent. Question 4: 1 = probable survivor without severe disabilities, 2 = probable survivor with severe disabilities, 3 = probable nonsurvivor