High-Throughput Deep Learning Detection of Mitral Regurgitation

Abstract

Background: Diagnosis of mitral regurgitation (MR) requires careful evaluation by echocardiography with Doppler imaging. This study presents the development and validation of a fully automated deep learning pipeline for identifying apical 4-chamber view videos with color Doppler echocardiography and detecting clinically significant (moderate or severe) MR from transthoracic echocardiograms.

Methods: A total of 58 614 transthoracic echocardiograms (2 587 538 videos) from Cedars-Sinai Medical Center were used to develop and test an automated pipeline to identify apical 4-chamber view videos with color Doppler across the mitral valve and then assess MR severity. The model was tested internally on a test set of 1800 studies (80 833 videos) from Cedars-Sinai Medical Center and externally evaluated in a geographically distinct cohort of 915 studies (46 890 videos) from Stanford Healthcare.

Results: In the held-out Cedars-Sinai Medical Center test set, the view classifier demonstrated an area under the curve (AUC) of 0.998 (0.998-0.999) and correctly identified 3452 of 3539 echocardiography videos as having color Doppler information across the mitral valve (sensitivity of 0.975 [0.968-0.982] and specificity of 0.999 [0.999-0.999] compared with manually curated videos). In the external test cohort from Stanford Healthcare, the view classifier correctly identified 1051 of 1055 manually curated videos with color Doppler information across the mitral valve (sensitivity of 0.996 [0.990-1.000] and specificity of 0.999 [0.999-0.999]). In the Cedars-Sinai Medical Center test cohort, MR moderate or greater in severity was detected with an AUC of 0.916 (0.899-0.932) and severe MR was detected with an AUC of 0.934 (0.913-0.953). In the Stanford Healthcare test cohort, the model detected MR moderate or greater in severity with an AUC of 0.951 (0.924-0.973) and severe MR with an AUC of 0.969 (0.946-0.987).

Conclusions: In this study, a novel automated pipeline for identifying clinically significant MR from full transthoracic echocardiography studies demonstrated excellent performance across large numbers of studies and across multiple institutions. Such an approach has the potential for automated screening and surveillance of MR.

Keywords: artificial intelligence; deep learning; echocardiography; mitral valve insufficiency.

Figure 1.

Computer Vision–Based Mitral Regurgitation Detection. An automated deep learning pipeline was trained to detect and stratify mitral regurgitation (MR) severity using large-scale data consisting of apical 4-chamber echocardiogram videos with color Doppler across the mitral valve (Cedars-Sinai Medical Center). The automated pipeline showed strong and consistent performance in test sets at Cedars-Sinai Medical Center and Stanford Healthcare. These results show that deep learning can accurately detect clinically significant MR using single-view transthoracic echocardiogram videos with Doppler information. Deep learning–based MR detection tools could serve as a part of point-of-care ultrasound screening as part of clinic visits or in resource-limited settings where imaging may be obtained by individuals with minimal training.

Figure 2.

Cedars-Sinai Medical Center and Stanford Data Set Isolation. A total of 34 714 color Doppler apical 4-chamber (A4C) videos were isolated from a larger set of videos from Cedars-Sinai Medical Center. A view classifier was trained and used to isolate A4C mitral Doppler videos from 915 studies containing 1055 suitable videos from Stanford Healthcare. The mitral regurgitation (MR) classification model was then benchmarked on an internal test set from Cedars-Sinai Medical Center and an external test set from Stanford Healthcare.

Figure 3.

Model Performance Across Severity and Institution. A, Receiver operating characteristic curves for detection of severe or moderate or greater mitral regurgitation (MR) at Cedars-Sinai Medical Center (CSMC) and Stanford Healthcare (SHC). Moderate includes moderate, moderate to severe, and severe MR. B and C, MR classification on test set videos from CSMC and SHC, respectively. Confusion matrix color map values were scaled on the basis of the proportion of actual disease cases in each class that were predicted in each possible disease category. This was done to allow for relative comparison of model performance across disease classes (none, mild, moderate, and severe) given class imbalance. AUC indicates area under the curve.

Figure 4.

Saliency Map Visualization for Mitral Regurgitation Classification Models. Echocardiogram videos with severe mitral regurgitation (MR) from Cedars-Sinai Medical Center (CSMC; top left) and Stanford Healthcare (SHC; bottom left) are shown on the left; videos with no MR from CSMC (top right) and SHC (bottom right) are shown on the right. Saliency maps were computed using the integrated gradients method. A final 2-dimensional heatmap was generated by using the maximum value along the temporal axis for each pixel location in the video. Brighter and closer to yellow pixels were more salient to model predictions; darker pixels were less important to the final prediction of the model. Severe MR was assessed by using the activation function for severe disease to generate a heatmap. When assessing controls, heatmaps were generated by stacking heatmaps for severe and moderate classes and taking the maximum between the 2 at each pixel location.

Figure 5.

Tracking Mitral Regurgitation Severity Across Serial Studies. Sankey diagram of patients with ≥4 echocardiogram studies with severity of mitral regurgitation (MR) tracked over time as evaluated by clinicians and by artificial intelligence model. Progression and change demonstrated by flow diagrams moving across MR severity.