Multinational Federated Learning Approach to Train ECG and Echocardiogram Models for Hypertrophic Cardiomyopathy Detection

Abstract

Background: Novel targeted treatments increase the need for prompt hypertrophic cardiomyopathy (HCM) detection. However, its low prevalence (0.5%) and resemblance to common diseases present challenges that may benefit from automated machine learning-based approaches. We aimed to develop machine learning models to detect HCM and to differentiate it from other cardiac conditions using ECGs and echocardiograms, with robust generalizability across multiple cohorts.

Methods: Single-institution HCM ECG models were trained and validated on external data. Multi-institution models for ECG and echocardiogram were trained on data from 3 academic medical centers in the United States and Japan using a federated learning approach, which enables training on distributed data without data sharing. Models were validated on held-out test sets for each institution and from a fourth academic medical center and were further evaluated for discrimination of HCM from aortic stenosis, hypertension, and cardiac amyloidosis. Last, automated detection was compared with manual interpretation by 3 cardiologists on a data set with a realistic HCM prevalence.

Results: We identified 74 376 ECGs for 56 129 patients and 8392 echocardiograms for 6825 patients at the 4 academic medical centers. Although ECG models trained on data from each institution displayed excellent discrimination of HCM on internal test data (C statistics, 0.88-0.93), the generalizability was limited, most notably for a model trained in Japan and tested in the United States (C statistic, 0.79-0.82). When trained in a federated manner, discrimination of HCM was excellent across all institutions (C statistics, 0.90-0.96 and 0.90-0.96 for ECG and echocardiogram model, respectively), including for phenotypic subgroups. The models further discriminated HCM from hypertension, aortic stenosis, and cardiac amyloidosis (C statistics, 0.84, 0.83, and 0.88, respectively, for ECG and 0.93, 0.94, 0.85, respectively, for echocardiogram). Analysis of electrocardiography-echocardiography paired data from 11 823 patients from an external institution indicated a higher sensitivity of automated HCM detection at a given positive predictive value compared with cardiologists (0.98 versus 0.81 at a positive predictive value of 0.01 for ECG and 0.78 versus 0.59 at a positive predictive value of 0.24 for echocardiogram).

Conclusions: Federated learning improved the generalizability of models that use ECGs and echocardiograms to detect and differentiate HCM from other causes of hypertrophy compared with training within a single institution.

Keywords: cardiomyopathy, hypertrophic; echocardiography; electrocardiography; machine learning.

Figure 1.

Study overview. A, The ECG–hypertrophic cardiomyopathy (HCM) models trained on data from a single institution discriminated HCM excellently on a held-out data set from that same institution, but some models generalized poorly on an external data set. B, Schematic of the process of federated learning. Multi-institutional models can be trained without data leaving any institution. In our case, ECG and echocardiogram models trained using federated learning not only discriminated HCM well on held-out test sets but also had excellent discrimination on external validation data sets from an independent institution. Schematic of deployment simulation. C, A cohort with an HCM prevalence of 0.5% was constructed to reflect prevalence in the general population. The stepwise approach with ECG followed by echocardiogram model achieved a sensitivity of 0.84 at a positive predictive value (PPV) of 0.25, whereas expert cardiologists could achieve a sensitivity of only 0.59 at a PPV of 0.24 even by performing echocardiograms on all patients in this cohort. AUROC indicates area under the receiver-operating characteristics curve; B, Brigham and Women’s Hospital; K, Keio University Hospital; M, Massachusetts General Hospital; and U, University of California San Francisco.

Figure 2.

Heterogeneity of ECG and AUROCs of models trained at individual institutions. A, Uniform Manifold Approximation and Projection (UMAP) projection of raw electrocardiographic recording stratified by institution. B, Heat map showing the performance of models trained at individual institutions. Held-out test data sets were used to evaluate model performance. The models were trained using 5-fold cross-validation for each institution, and all models were tested on their own institution test set along with 3 external data sets. The area under the receiver-operating characteristics curve (AUCROC) and 95% CI based on the SE for the 5 models are shown. BWH indicates Brigham and Women’s Hospital; Keio, Keio University Hospital; MGH, Massachusetts General Hospital; and UCSF, University of California San Francisco.

Figure 3.

Discrimination of HCM by the ECG model trained with federated learning. A, Receiver-operating characteristics (ROC) plots for hypertrophic cardiomyopathy (HCM) discrimination of the ECG model trained in a federated manner on a held-out internal test data set for each institution (3569, 1375, and 2656 patients in test data set from Massachusetts General Hospital [MGH], University of California San Francisco [UCSF], and Keio University Hospital [Keio], respectively) and on an (B) external data set (18 118 patients from Brigham and Women’s Hospital [BWH]). C, ROC curves for discriminating HCM with and without outflow tract obstruction (17 830 and 17 769 patients, respectively) and apical and nonapical HCM (17 767 and 18 022 patients, respectively). D, ROC curves for discriminating HCM with hypertension, aortic valve stenosis (AS), or cardiac amyloidosis (1020, 746, and 811 patients, respectively). E, An ROC curve for discrimination of HCM before developing HCM (17 760 patients). The 95% CI of the true-positive fraction for a given false-positive fraction is shown as a blue ribbon (N is the number of studies). F, Gradient-weighted class activation mapping images for HCM samples with and without high voltage. Areas of primary focus of the model are indicated by black arrowheads. AUC indicates area under the curve; HTN, hypertension; and LVH, left ventricular hypertrophy.

Figure 4.

Discrimination of HCM by the echocardiogram model trained with federated learning. A, Receiver-operating characteristics (ROC) plots for detecting hypertrophic cardiomyopathy (HCM) using echocardiogram model trained in a federated manner tested on held-out internal data for each institution (1700, 1031, and 1639 patients in test data set from Massachusetts General Hospital [MGH], University of California San Francisco [UCSF], and Keio University Hospital [Keio], respectively) and on an (B) external data set (2455 patients from Brigham and Women’s Hospital [BWH]). C, ROC plots for detecting HCM with and without outflow tract obstruction (2253 and 2205 patients, respectively) and apical and nonapical HCM (2192 and 2391 patients, respectively). D, ROC curves for discriminating HCM with hypertension (HTN), aortic valve stenosis (AS), cardiac amyloidosis (1491, 611, and 640 patients, respectively). E, ROC curve for discrimination of HCM before developing HCM (2403 patients). The 95% CI of the true-positive fraction for a given false-positive fraction is shown as a blue ribbon (N is the number of studies). F, Gradient-weighted class activation mapping images for HCM sample. Areas of primary focus of the model are indicated by white arrowheads. AUC indicates area under the curve; and LVH, left ventricular hypertrophy.

Figure 5.

Deployment simulation of the models on surveillance populations for detecting HCM. A, Precision recall curve (PRC) and receiver-operating characteristics (ROC) plots for the ECG model and echocardiography model for discrimination of patients with hypertrophic cardiomyopathy (HCM) in the surveillance populations. The 95% CIs of precision and true-positive fraction are shown as blue ribbons in the PRC and ROC curves, respectively (N is the number of patients). The sensitivity, specificity, and positive predictive value (PPV) for detecting patients with HCM by human experts are plotted with the curves. B, PRC curves for the stepwise approach applying echocardiogram model after prescreening with ECG model using 2 cutoffs corresponding to the PPV of the any abnormal ECG findings by human experts. The overall sensitivity, specificity, and PPV for detecting patients with HCM by human experts are plotted. Overall recall is the number of HCMs detected after all the processes divided by the total number of HCM cases in the original cohort. AUC indicates area under the curve; and LVH, left ventricular hypertrophy.