Combining chest X-rays and electronic health record (EHR) data using machine learning to diagnose acute respiratory failure

Abstract

Objective: When patients develop acute respiratory failure (ARF), accurately identifying the underlying etiology is essential for determining the best treatment. However, differentiating between common medical diagnoses can be challenging in clinical practice. Machine learning models could improve medical diagnosis by aiding in the diagnostic evaluation of these patients.

Materials and methods: Machine learning models were trained to predict the common causes of ARF (pneumonia, heart failure, and/or chronic obstructive pulmonary disease [COPD]). Models were trained using chest radiographs and clinical data from the electronic health record (EHR) and applied to an internal and external cohort.

Results: The internal cohort of 1618 patients included 508 (31%) with pneumonia, 363 (22%) with heart failure, and 137 (8%) with COPD based on physician chart review. A model combining chest radiographs and EHR data outperformed models based on each modality alone. Models had similar or better performance compared to a randomly selected physician reviewer. For pneumonia, the combined model area under the receiver operating characteristic curve (AUROC) was 0.79 (0.77-0.79), image model AUROC was 0.74 (0.72-0.75), and EHR model AUROC was 0.74 (0.70-0.76). For heart failure, combined: 0.83 (0.77-0.84), image: 0.80 (0.71-0.81), and EHR: 0.79 (0.75-0.82). For COPD, combined: AUROC = 0.88 (0.83-0.91), image: 0.83 (0.77-0.89), and EHR: 0.80 (0.76-0.84). In the external cohort, performance was consistent for heart failure and increased for COPD, but declined slightly for pneumonia.

Conclusions: Machine learning models combining chest radiographs and EHR data can accurately differentiate between common causes of ARF. Further work is needed to determine how these models could act as a diagnostic aid to clinicians in clinical settings.

Keywords: acute respiratory failure; chest X-ray; electronic health record; machine learning.

Figure 1.

Performance of the combined, image, EHR model for diagnosis of pneumonia, heart failure, and COPD in the internal and external cohorts. Model performance evaluated based on the area under the receiver operator characteristic curve (AUROC). Black horizontal lines indicate median performance for each model. When the models were evaluated using diagnosis based on chart reviews in the internal cohort, the combined model outperforms the image and EHR models on most data splits in terms of AUROC for identifying pneumonia, heart failure, and COPD (A). Model performance decreased for pneumonia and COPD when evaluated using discharge diagnosis codes and medications (B). Model performance on the external cohort was evaluated using discharge diagnosis codes and medications (C) and was similar or better compared to the internal cohort (B) with the exception of pneumonia. The combined model consistently outperformed the other models across cohorts in terms of macro-average AUROC which combines model performance across all 3 diagnoses. COPD: chronic obstructive pulmonary disease.

Figure 2.

Chest radiograph heatmaps in patients where the model correctly diagnosed pneumonia, heart failure, or COPD with high probability. The overlaying heatmap generated by Grad-CAM highlights the regions the model focused on when estimating the likely diagnosis (blue: low contribution, yellow: high contribution). For both the image and combined models, the model looked at the lungs and the heart when diagnosing pneumonia and heart failure, and the trachea when diagnosing COPD. Heatmaps were normalized on individual images to highlight the most important areas of each image, therefore heatmap values should not be compared across images. Image processing was performed, including histogram equalization to increase contrast in the original images, and then images were resized to 512 × 512 pixels.