Comprehensive clinical application analysis of artificial intelligence-enabled electrocardiograms for screening multiple valvular heart diseases

Abstract

Background: Valvular heart disease (VHD) is becoming increasingly important to manage the risk of future complications. Electrocardiographic (ECG) changes may be related to multiple VHDs, and (AI)-enabled ECG has been able to detect some VHDs. We aimed to develop five deep learning models (DLMs) to identify aortic stenosis, aortic regurgitation, pulmonary regurgitation, tricuspid regurgitation, and mitral regurgitation.

Methods: Between 2010 and 2021, 77,047 patients with echocardiography and 12-lead ECG performed within 7 days were identified from an academic medical center to provide DLM development (122,728 ECGs), and internal validation (7,637 ECGs). Additional 11,800 patients from a community hospital were identified to external validation. The ECGs were classified as with or without moderate-to-severe VHDs according to transthoracic echocardiography (TTE) records, and we also collected the other echocardiographic data and follow-up TTE records to identify new-onset valvular heart diseases.

Results: AI-ECG adjusted for age and sex achieved areas under the curves (AUCs) of >0.84, >0.80, >0.77, >0.83, and >0.81 for detecting aortic stenosis, aortic regurgitation, pulmonary regurgitation, tricuspid regurgitation, and mitral regurgitation, respectively. Since predictions of each DLM shared similar components of ECG rhythms, the positive findings of each DLM were highly correlated with other valvular heart diseases. Of note, a total of 37.5-51.7% of false-positive predictions had at least one significant echocardiographic finding, which may lead to a significantly higher risk of future moderate-to-severe VHDs in patients with initially minimal-to-mild VHDs.

Conclusion: AI-ECG may be used as a large-scale screening tool for detecting VHDs and a basis to undergo an echocardiography.

Keywords: artificial intelligence; deep learning; electrocardiogram; transthoracic echocardiography; valvular heart disease.

Figure 1

ROC curve analysis for VHD from a DLM based on age, sex, and ECG voltage–time traces. The receiver operating characteristic (ROC) curve (x-axis = specificity and y-axis = sensitivity) and area under the ROC curve (AUC) were calculated using the internal validation set (A) and external validation set (B). The operating point was selected based on the maximum Youden’s index in the tuning set, which was used for calculating the corresponding sensitivities and specificities in the two validation sets.

Figure 2

The components of AI predictions for detecting each valvular disease. (A) Relationship between ECG-screened valvular diseases and ECG rhythms. The plots display two groups, positive (AI-positive) and negative (AI-negative) findings, by the ECG networks using ECG alone. Sinus rhythm is associated with AI-negative (green bar), and other abnormal rhythms are associated with AI-positive (red bar). Abbreviations: ^*p < 0.05; ^**p < 0.01; ^***p < 0.001. The +/− demonstrates the positive/negative relationship. (B) The relationship between each valvular disease in actual status and prediction. The values in each cell are the Spearman correlation coefficients.

Figure 3

Prevalence (p) of echocardiographic abnormalities in patients stratified by each AI classification using ECG alone. The plots display the abnormal prevalence in the two groups, including positive and negative findings based on ECG. The ≥1 of valvular diseases was defined as at least 1 moderate-to-severe valvular disease, and the ≥1 of significant findings was defined as at least 1 abnormal echocardiographic finding. The relative risk (RR) was calculated as (p_AI-positive/p_AI-negative) and is presented with the associated 95% confidence interval.

Figure 4

Long-term incidence of developing severity stratified by AI classification using ECG alone. Long-term incidence of developing each moderate-to-severe valvular disease in patients with initially minimal-to-mild valvular diseases stratified by AI classification using ECG alone. Long-term outcome of patients with echocardiographic minimal-to-mild valvular diseases at the time of initial classification, stratified by the initial network classification. The ordinate shows the cumulative incidence of developing moderate-to-severe valvular diseases, and the abscissa indicates years from the time of index ECG–TTE evaluation. A significantly higher risk of future moderate-to-severe valvular diseases was present when the AI algorithm defined the ECG as positive compared with patients with minimal-to-mild valvular diseases who were classified as having a negative finding by the ECG network. The analyses were conducted in both internal and external validation sets. The table shows the at-risk population and cumulative risk for the given time intervals in each risk stratification.

Figure 5

Development, tuning, internal validation, and external validation set generation and ECG labeling of VHD. Schematic of the dataset creation and analysis strategy, which was devised to assure a robust and reliable dataset for training, validating, and testing of the network. Once a patient’s data were placed in one of the datasets, that individual’s data were used only in that set, avoiding ‘cross-contamination’ among the training, validation, and test datasets. The details of the flow chart and how each of the datasets was used are described in the Methods.