Detecting structural heart disease from electrocardiograms using AI

Abstract

Early detection of structural heart disease is critical to improving outcomes, but widespread screening remains limited by the cost and accessibility of imaging tools such as echocardiography^1,2. Recent advances in machine learning applied to heart rhythm recordings have shown promise in identifying disease^3,4, although previous work has been limited by development in narrow populations or targeting only select heart conditions⁵. Here we introduce a deep learning model, EchoNext, trained on more than 1 million heart rhythm and imaging records across a large and diverse health system to detect many forms of structural heart disease. The model demonstrated high diagnostic accuracy in internal and external validation, outperforming cardiologists in a controlled evaluation and showing consistent performance across different care settings and racial and/or ethnic groups. The models were prospectively evaluated in a clinical trial of patients without previous cardiac imaging, successfully identifying previously undiagnosed heart disease. These findings support the potential of artificial intelligence to expand access to heart disease screening at scale. To enable further development and transparency, we have publicly released model weights and a large, annotated dataset linking heart rhythm data to imaging-based diagnoses.

Fig. 1. Model development: NYP multicentre cohort derivation.

The deep learning model was trained and tested using data from an eight-hospital system (NYP Hospital). ECG data were accessed using the MUSE system with removal of ECGs with missing age, sex and patient identifier, poor study quality designation by machine recommending repeating of ECG or presence of ventricular pacing. Echocardiogram data were accessed using hospital systems with removal of patients with repaired or replaced heart failures. This yielded 1.2 million ECG–echocardiogram pairs in 230,018 unique patients with data split into train, validation and test sets.

Fig. 2. Multicentre EchoNext performance.

Performance of the model in detection of individual and compositive SHDs. a,b, By AUROC (a) and AUPRC (b), the model had high performance in detection of SHD in the internal eight-hospital NYP system test set and three geographically distinct external test sets (Montreal Heart, Cedars-Sinai and University of California San Francisco (UCSF)). c, Individual disease models had the highest performance in the detection of reduced low left ventricular (LV) and right ventricular (RV) systolic function by AUROC with favourable performance for other disease states. d, Assessment of the AUPRC for the individual disease states is highly dependent on the underlying prevalence of the individual disease states. TR, tricuspid regurgitation. Dashed lines (a,c) indicate random classifier.

Fig. 3. Performance characteristics of EchoNext in retrospective validation, comparison to cardiologists, silent deployment, and clinical trial.

a, In test sets of held-out patients at these sites, as well as in three geographically distinct external test sets, the model demonstrated high accuracy, as demonstrated by AUROC. Dashed line indicates random classifier. b, In a survey of ECGs shown to cardiologists to assess for the presence of SHD, the AI model demonstrated superior performance in SHD detection compared with cardiologists alone or cardiologists given the EchoNext risk score (n = 3,200 cardiologist interpretations). Error bars show the CIs derived from results across 13 cardiologists; because the AI model was run once on the entire set of 150 ECGs, there are no error bars for the ‘AI alone’ results. c, This model was evaluated in a temporally distinct held-out set with similar accuracy, with 45% (n = 3,444) of patients labelled as high risk by the model failing to undergo echocardiography as part of routine clinical care. d, The clinical use of AI-ECG to detect SHD was evaluated in a single-arm, single-site, open-label pilot clinical trial, DISCOVERY, with stratified recruitment of patients (N = 100) with an ECG but no previous echocardiogram. This trial used a related ECG model, ValveNet, which was trained to detect left-sided VHD (Left-VHD) with patients selected stratified sampling by their AI-ECG scores. This trial showed a high-level of discrimination in the detection of Left-VHD (primary endpoint) and SHD (secondary endpoint), and post hoc assessment using the second-generation EchoNext model demonstrated an even greater degree of risk stratification with 73% of patients in the highest risk and 6% of patients in the lowest-risk groups being found to have SHD. Left-VHD, moderate or severe aortic stenosis, aortic regurgitation or mitral regurgitation; SHD, LVEF less than or equal to 45%, low left ventricular wall thickness greater than or equal to 1.3 cm, moderate or severe right ventricular dysfunction, any moderate or severe VHD, PASP greater than or equal to 45 mm Hg or a moderate or large pericardial effusion.

Fig. 4. Characteristics of the released Columbia ECG dataset and performance of the Columbia mini-model trained on these data for SHD prediction.

A 100,000 ECG dataset is being released from data from Columbia University Irving Medical Center as part of this paper. These data consist of the ECG waveform, ECG tabular features and paired echocardiographic data. A model trained and tested on this dataset demonstrated similar performance in SHD detection as the multisite model that served as the primary analysis in this study when assessed by AUROC and AUPRC. Dashed line indicates random classifier.