Validation study of an automated electronic acute lung injury screening tool

Abstract

OBJECTIVE The authors designed an automated electronic system that incorporates data from multiple hospital information systems to screen for acute lung injury (ALI) in mechanically ventilated patients. The authors evaluated the accuracy of this system in diagnosing ALI in a cohort of patients with major trauma, but excluding patients with congestive heart failure (CHF). DESIGN Single-center validation study. Arterial blood gas (ABG) data and chest radiograph (CXR) reports for a cohort of intensive care unit (ICU) patients with major trauma but excluding patients with CHF were screened prospectively for ALI requiring intubation by an automated electronic system. The system was compared to a reference standard established through consensus of two blinded physician reviewers who independently screened the same population for ALI using all available ABG data and CXR images. The system's performance was evaluated (1) by measuring the sensitivity and overall accuracy, and (2) by measuring concordance with respect to the date of ALI identification (vs. reference standard). MEASUREMENTS One hundred ninety-nine trauma patients admitted to our level 1 trauma center with an initial injury severity score (ISS) >/= 16 were evaluated for development of ALI in the first five days in an ICU after trauma. Main RESULTS The system demonstrated 87% sensitivity (95% confidence interval [CI] 82.3-91.7) and 89% specificity (95% CI 84.7-93.4). It identified ALI before or within the 24-hour period during which ALI was identified by the two reviewers in 87% of cases. CONCLUSIONS An automated electronic system that screens intubated ICU trauma patients, excluding patients with CHF, for ALI based on CXR reports and results of ABGs is sufficiently accurate to identify many early cases of ALI.

Figure 1

Diagnostic algorithm followed by the Acute Respiratory Distress Syndrome Network research coordinator to identify patients with ALI. ICU, intensive care unit; MICU, medical intensive care unit; CCU, cardiac care unit; SICU, surgical intensive care unit; P/F ratio, PaO₂/FiO₂ ratio; ABG, arterial blood gas; (H) hours; CXR, chest x-ray; ALI, acute lung injury. *Assuming no clinical evidence of left atrial hypertension (left-sided congestive heart failure) or, if available, that the pulmonary arterial occlusion pressure was not 18 mm Hg or greater.

Figure 2

Concordance data for day of identification of acute lung injury (ALI) for the 46 cases of ALI that were diagnosed both by the automated system and the two physician reviewers. On the ordinate is the percent of cases identified. On the abscissa is the day of identification of ALI by the automated system, compared to the day of identification by the two physician reviewers. A case was considered to have been identified on the same day if it was identified by both methods within a 24-hour period. If a case was identified by the two methods outside of this 24-hour window, the disparity was quantified by counting the number of 24-hour windows between the two dates.