Augmenting existing deterioration indices with chest radiographs to predict clinical deterioration

Abstract

Importance: When hospitals are at capacity, accurate deterioration indices could help identify low-risk patients as potential candidates for home care programs and alleviate hospital strain. To date, many existing deterioration indices are based entirely on structured data from the electronic health record (EHR) and ignore potentially useful information from other sources.

Objective: To improve the accuracy of existing deterioration indices by incorporating unstructured imaging data from chest radiographs.

Design, setting, and participants: Machine learning models were trained to predict deterioration of patients hospitalized with acute dyspnea using existing deterioration index scores and chest radiographs. Models were trained on hospitalized patients without coronavirus disease 2019 (COVID-19) and then subsequently tested on patients with COVID-19 between January 2020 and December 2020 at a single tertiary care center who had at least one radiograph taken within 48 hours of hospital admission.

Main outcomes and measures: Patient deterioration was defined as the need for invasive or non-invasive mechanical ventilation, heated high flow nasal cannula, IV vasopressor administration or in-hospital mortality at any time following admission. The EPIC deterioration index was augmented with unstructured data from chest radiographs to predict risk of deterioration. We compared discriminative performance of the models with and without incorporating chest radiographs using area under the receiver operating curve (AUROC), focusing on comparing the fraction and total patients identified as low risk at different negative predictive values (NPV).

Results: Data from 6278 hospitalizations were analyzed, including 5562 hospitalizations without COVID-19 (training cohort) and 716 with COVID-19 (216 in validation, 500 in held-out test cohort). At a NPV of 0.95, the best-performing image-augmented deterioration index identified 49 more (9.8%) individuals as low-risk compared to the deterioration index based on clinical data alone in the first 48 hours of admission. At a NPV of 0.9, the EPIC image-augmented deterioration index identified 26 more individuals (5.2%) as low-risk compared to the deterioration index based on clinical data alone in the first 48 hours of admission.

Conclusion and relevance: Augmenting existing deterioration indices with chest radiographs results in better identification of low-risk patients. The model augmentation strategy could be used in the future to incorporate other forms of unstructured data into existing disease models.

Fig 1. Image-augmented deterioration model.

The image-augmented deterioration index combines outputs from an image model, an EHR deterioration index, and two time dependent variables encoding the time since the chest x-ray was performed and the time since the patient was admitted. Both the EHR deterioration index and image model can be varied. We provide additional details of chest x-ray preprocessing in the S1 Fig.

Fig 2. Prediction timeline during a hospitalization.

A hospitalization timeline is shown for a hypothetical patient illustrating typical data availability and when predictions can occur. Image model predictions are made only during the time at which image studies are taken. The image-augmented deterioration index makes predictions for all windows after the image has been taken, using the deterioration index prediction prior to the first radiograph.

Fig 3. Model predictions for COVID19 positive patients within the first 48 hours of admission, shown with exponential weight moving average and 95% CIs.

Each plot shows the number of patients flagged as low-risk by lowest aggregated prediction and the resulting accuracy for that fraction of patients. The top plot compares the EDI augmented model to the EDI model. The bottom plot compares the MCURES augmented model to the MCURES model.

Fig 4. The fraction of patients correctly identified by each of the models as low-risk, shown over hours after hospital admission.

Note that the MCURES and MCURES-augmented models generally have better performance over the EDI and EDI-augmented models and so we select 0.9 NPV as an appropriate cutoff for EDI and 0.95 NPV for MCURES.