Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Sep 17;150(12):911-922.
doi: 10.1161/CIRCULATIONAHA.124.068996. Epub 2024 Jun 17.

Deep Learning for Echo Analysis, Tracking, and Evaluation of Mitral Regurgitation (DELINEATE-MR)

Affiliations

Deep Learning for Echo Analysis, Tracking, and Evaluation of Mitral Regurgitation (DELINEATE-MR)

Aaron Long et al. Circulation. .

Abstract

Background: Artificial intelligence, particularly deep learning (DL), has immense potential to improve the interpretation of transthoracic echocardiography (TTE). Mitral regurgitation (MR) is the most common valvular heart disease and presents unique challenges for DL, including the integration of multiple video-level assessments into a final study-level classification.

Methods: A novel DL system was developed to intake complete TTEs, identify color MR Doppler videos, and determine MR severity on a 4-step ordinal scale (none/trace, mild, moderate, and severe) using the reading cardiologist as a reference standard. This DL system was tested in internal and external test sets with performance assessed by agreement with the reading cardiologist, weighted κ, and area under the receiver-operating characteristic curve for binary classification of both moderate or greater and severe MR. In addition to the primary 4-step model, a 6-step MR assessment model was studied with the addition of the intermediate MR classes of mild-moderate and moderate-severe with performance assessed by both exact agreement and ±1 step agreement with the clinical MR interpretation.

Results: A total of 61 689 TTEs were split into train (n=43 811), validation (n=8891), and internal test (n=8987) sets with an additional external test set of 8208 TTEs. The model had high performance in MR classification in internal (exact accuracy, 82%; κ=0.84; area under the receiver-operating characteristic curve, 0.98 for moderate or greater MR) and external test sets (exact accuracy, 79%; κ=0.80; area under the receiver-operating characteristic curve, 0.98 for moderate or greater MR). Most (63% internal and 66% external) misclassification disagreements were between none/trace and mild MR. MR classification accuracy was slightly higher using multiple TTE views (accuracy, 82%) than with only apical 4-chamber views (accuracy, 80%). In subset analyses, the model was accurate in the classification of both primary and secondary MR with slightly lower performance in cases of eccentric MR. In the analysis of the 6-step classification system, the exact accuracy was 80% and 76% with a ±1 step agreement of 99% and 98% in the internal and external test set, respectively.

Conclusions: This end-to-end DL system can intake entire echocardiogram studies to accurately classify MR severity and may be useful in helping clinicians refine MR assessments.

Keywords: artificial intelligence; deep learning; echocardiography; mitral valve insufficiency.

PubMed Disclaimer

Conflict of interest statement

Dr Poterucha owns stock in Abbott Laboratories and Baxter International with research support provided to his institution from the Amyloidosis Foundation, American Heart Association (awards 933452 and 23SCISA1077494; https://doi.org/10.58275/AHA.23SCISA1077494.pc.gr.172160), Eidos Therapeutics, Pfizer, Edwards Lifesciences, and Janssen. Dr Hahn reports speaker fees from Abbott Structural, Baylis Medical, Edwards Lifesciences, Medtronic, Philips Healthcare, and Siemens Healthineers; she has institutional consulting contracts for which she receives no direct compensation with Abbott Structural, Anteris, Edwards Lifesciences, Medtronic and Novartis; she is chief scientific officer for the Echocardiography Core Laboratory at the Cardiovascular Research Foundation for multiple industry-sponsored valve trials, for which she receives no direct industry compensation. Dr Einstein reports receiving authorship fees from Wolters Kluwer Healthcare—UpToDate and serving on a scientific advisory board for Canon Medical Systems; his institution has grants/grants pending from Attralus, BridgeBio, Canon Medical Systems, GE HealthCare, Intellia Therapeutics, Ionis Pharmaceuticals, Neovasc, Pfizer, Roche Medical Systems, and W.L. Gore & Associates. The other authors report no conflicts.

Figures

Figure 1.
Figure 1.
Deep learning classification of MR from complete echocardiograms. In this study, an end-to-end deep learning system was trained to classify the severity of mitral regurgitation (MR). The system intakes an entire transthoracic echocardiogram (TTE) study, identifies color Doppler clips that assess the mitral valve, makes a clip-level MR classification, and combines those clip-level classifications into a study-level MR classification on a 4-step scale of none/trace, mild, moderate, and severe. This system demonstrated high accuracy in internal and external test sets with exact agreement between the deep learning model of 79% to 82% and weighted κ coefficients of 0.80 to 0.84. There was strong binary classification of moderate or greater MR with area under the receiver-operating characteristic curve (AUROC) of 0.98 for the detection of moderate or greater MR. Furthermore, the primary deep learning system that integrates multiple TTE views had superior performance compared with an MR classification model using only the 4-chamber view. In the 1% of cases in the internal test set with the most significant disagreement between the TTE report and the deep learning MR classification, an expert panel review indicated that the interpreting cardiologist had misclassified the MR severity in 29% of those cases. These findings indicate that this deep learning system may be useful for improving and monitoring MR classification by echocardiography. AI indicates artificial intelligence.
Figure 2.
Figure 2.
Patient flow. This figure demonstrates the internal and external data sets used to train, validate, and test the mitral regurgitation deep learning (DL) model. LVAD indicates left ventricular assist device; MV, mitral valve; and TTE, transthoracic echocardiography.
Figure 3.
Figure 3.
Model confusion matrices in internal and external test sets. This figure demonstrates the confusion matrix in the internal (A) and external (B) test sets comparing the cardiologist clinical interpretation (vertical axis) and the DL model prediction (horizontal axis) for mitral regurgitation on a 4-step ordinal scale. Exact agreement between the DL mode and cardiologist was present in 82% in the internal test set and 79% of the external test set. AI indicates artificial intelligence.
Figure 4.
Figure 4.
Receiver-operating characteristic and precision-recall curves for the detection of moderate or greater MR. These figures demonstrate the model performance in the detection of mitral regurgitation (MR) in the internal (black) and external (red) test sets. A, Receiver operator characteristic (ROC) curve for at least moderate MR and the ROC curve for severe MR (B). C and D, The precision-recall (positive-predictive value–sensitivity) curves for detection of at least moderate MR (C) and severe MR (D). AU-PRC indicates area under the precision-recall curve; and AUROC, area under the receiver operator characteristic curve.

Comment in

References

    1. Nkomo VT, Gardin JM, Skelton TN, Gottdiener JS, Scott CG, Enriquez-Sarano M. Burden of valvular heart diseases: a population-based study. Lancet. 2006;368:1005–1011. doi: 10.1016/S0140-6736(06)69208-8 - PubMed
    1. Coffey S, Roberts-Thomson R, Brown A, Carapetis J, Chen M, Enriquez-Sarano M, Zuhlke L, Prendergast BD. Global epidemiology of valvular heart disease. Nat Rev Cardiol. 2021;18:853–864. doi: 10.1038/s41569-021-00570-z - PubMed
    1. d’Arcy JL, Coffey S, Loudon MA, Kennedy A, Pearson-Stuttard J, Birks J, Frangou E, Farmer AJ, Mant D, Wilson J, et al. Large-scale community echocardiographic screening reveals a major burden of undiagnosed valvular heart disease in older people: the OxVALVE Population Cohort Study. Eur Heart J. 2016;37:3515–3522. doi: 10.1093/eurheartj/ehw229 - PMC - PubMed
    1. Parcha V, Patel N, Kalra R, Suri SS, Arora G, Arora P. Mortality due to mitral regurgitation among adults in the United States: 1999-2018. Mayo Clin Proc. 2020;95:2633–2643. doi: 10.1016/j.mayocp.2020.08.039 - PubMed
    1. Dziadzko V, Clavel MA, Dziadzko M, Medina-Inojosa JR, Michelena H, Maalouf J, Nkomo V, Thapa P, Enriquez-Sarano M. Outcome and undertreatment of mitral regurgitation: a community cohort study. Lancet. 2018;391:960–969. doi: 10.1016/S0140-6736(18)30473-2 - PMC - PubMed

LinkOut - more resources