A novel interpretable deep learning model for diagnosis in emergency department dyspnoea patients based on complete data from an entire health care system

Abstract

Background: Dyspnoea is one of the emergency department's (ED) most common and deadly chief complaints, but frequently misdiagnosed and mistreated. We aimed to design a diagnostic decision support which classifies dyspnoeic ED visits into acute heart failure (AHF), exacerbation of chronic obstructive pulmonary disease (eCOPD), pneumonia and "other diagnoses" by using deep learning and complete, unselected data from an entire regional health care system.

Methods: In this cross-sectional study, we included all dyspnoeic ED visits of patients ≥ 18 years of age at the two EDs in the region of Halland, Sweden, 07/01/2017-12/31/2019. Data from the complete regional health care system within five years prior to the ED visit were analysed. Gold standard diagnoses were defined as the subsequent in-hospital or ED discharge notes, and a subsample was manually reviewed by emergency medicine experts. A novel deep learning model, the clinical attention-based recurrent encoder network (CareNet), was developed. Cohort performance was compared to a simpler CatBoost model. A list of all variables and their importance for diagnosis was created. For each unique patient visit, the model selected the most important variables, analysed them and presented them to the clinician interpretably by taking event time and clinical context into account. AUROC, sensitivity and specificity were compared.

Findings: The most prevalent diagnoses among the 10,315 dyspnoeic ED visits were AHF (15.5%), eCOPD (14.0%) and pneumonia (13.3%). Median number of unique events, i.e., registered clinical data with time stamps, per ED visit was 1,095 (IQR 459-2,310). CareNet median AUROC was 87.0%, substantially higher than the CatBoost model´s (81.4%). CareNet median sensitivity for AHF, eCOPD, and pneumonia was 74.5%, 92.6%, and 54.1%, respectively, with a specificity set above 75.0, slightly inferior to that of the CatBoost baseline model. The model assembled a list of 1,596 variables by importance for diagnosis, on top were prior diagnoses of heart failure or COPD, daily smoking, atrial fibrillation/flutter, life management difficulties and maternity care. Each patient visit received their own unique attention plot, graphically displaying important clinical events for the diagnosis.

Interpretation: We designed a novel interpretable deep learning model for diagnosis in emergency department dyspnoea patients by analysing unselected data from a complete regional health care system.

Fig 1. An ED patient visit.

A single ED patient visit in the health care system. The medical past is divided into periods and clinical contexts.

Fig 2. Hierarchical attention network of a patent visit.

The hierarchical attention network of CareNet analysing a patient visit. Circles represent vectors.

Fig 3. CareNet performance across different diagnostic labels.

CareNet AUROC using one year of data prior to index visit and expert labels. An illustrative example from one of the validation folds with the highest micro AUROC.

Fig 4

A-B. Individual patient attention plots with diagnosis AHF and pneumonia. 4A. Attention plot for a patient with AHF. The period 16–20 weeks prior to the index visit has the highest attention, i.e., is most important for diagnosis (diagram on top). During this most important period, the “others” category (diagram down to the left) is most important for diagnosis. The “others” variables are described in Table 2. Among the “others” variables, attention is highest for collected and, after that, prescribed medication of “R03 drugs for obstructive airway diseases” (diagram down to the right). 4B. Attention plot for a patient with pneumonia. The period 0–5 weeks prior to the index visit has the highest attention, i.e., is most important for diagnosis (diagram on top). During this most important period, the “others” category (diagram down to the left) is most important for diagnosis. The “others” variables are described in Table 2. Among the “others” variables, attention is highest for “stopped smoking > 6 months ago” and “collected medication R05 cough and cold preparations” (diagram down to the right).