Multimodal Deep Learning for ARDS Detection

Abstract

Objective: Poor outcomes in acute respiratory distress syndrome (ARDS) can be alleviated with tools that support early diagnosis. Current machine learning methods for detecting ARDS do not take full advantage of the multimodality of ARDS pathophysiology. We developed a multimodal deep learning model that uses imaging data, continuously collected ventilation data, and tabular data derived from a patient's electronic health record (EHR) to make ARDS predictions.

Materials and methods: A chest radiograph (x-ray), at least two hours of ventilator waveform (VWD) data within the first 24 hours of intubation, and EHR-derived tabular data were used from 220 patients admitted to the ICU to train a deep learning model. The model uses pretrained encoders for the x-rays and ventilation data and trains a feature extractor on tabular data. Encoded features for a patient are combined to make a single ARDS prediction. Ablation studies for each modality assessed their effect on the model's predictive capability.

Results: The trimodal model achieved an area under the receiver operator curve (AUROC) of 0.86 with a 95% confidence interval of 0.01. This was a statistically significant improvement (p<0.05) over single modality models and bimodal models trained on VWD+tabular and VWD+x-ray data.

Discussion and conclusion: Our results demonstrate the potential utility of using deep learning to address complex conditions with heterogeneous data. More work is needed to determine the additive effect of modalities on ARDS detection. Our framework can serve as a blueprint for building performant multimodal deep learning models for conditions with small, heterogeneous datasets.

Figure 1:

A schematic depiction of our trimodal architecture showing encoders, projection heads, and classification layers. X-ray and ventilator waveform encoders use a convolutional neural network architecture, while tabular encoders, projection heads, and classification networks use fully connected layers.

Figure 2:

A calibration curve (in orange) for our trimodal model showing predicted probability on the x-axis and true probability on the left y-axis. Overlayed with the calibration curve is the prediction distribution (in gray). The distribution shows the prediction frequency, on the right y-axis, at each probability decile, the x-axis.