Machine learning-enabled screening for aortic stenosis with handheld ultrasound

Abstract

Aims: Neural network classifiers can detect aortic stenosis (AS) using limited cardiac ultrasound images. While networks perform very well using cart-based imaging, they have never been tested or fine-tuned for use with focused cardiac ultrasound (FoCUS) acquisitions obtained on handheld ultrasound devices.

Methods and results: Prospective study performed at Tufts Medical Center. All patients ≥65 years of age referred for clinically indicated transthoracic echocardigraphy (TTE) were eligible for inclusion. Parasternal long axis and parasternal short axis imaging was acquired using a commercially available handheld ultrasound device. Our cart-based AS classifier (trained on ∼10 000 images) was tested on FoCUS imaging from 160 patients. The median age was 74 (inter-quartile range 69-80) years, 50% of patients were women. Thirty patients (18.8%) had some degree of AS. The area under the received operator curve (AUROC) of the cart-based model for detecting AS was 0.87 (95% CI 0.75-0.99) on the FoCUS test set. Last-layer fine-tuning on handheld data established a classifier with AUROC of 0.94 (0.91-0.97). AUROC during temporal external validation was 0.97 (95% CI 0.89-1.0). When performance of the fine-tuned AS classifier was modelled on potential screening environments (2 and 10% AS prevalence), the positive predictive value ranged from 0.72 (0.69-0.76) to 0.88 (0.81-0.97) and negative predictive value ranged from 0.94 (0.94-0.94) to 0.99 (0.99-0.99) respectively.

Conclusion: Our cart-based machine-learning model for AS showed a drop in performance when tested on handheld ultrasound imaging collected by sonographers. Fine-tuning the AS classifier improved performance and demonstrates potential as a novel approach to detecting AS through automated interpretation of handheld imaging.

Keywords: aortic stenosis; diagnosis; echocardiography; machine learning.

Graphical Abstract

Previously developed machine-learning aortic stenosis classifier was fine-tuned to work with handheld ultrasound imaging. The new optimized classifier can enable AS detection upstream of traditional echocardiography laboratories.

Figure 1

Receiver operator curves differentiating between no AS and any AS. Receiver operator curves in the test set. To obtain robust estimates of performance, we repeat all evaluations over five separate train/test splits of the 160 FoCUS scans in our dataset. Each split is drawn randomly and independently. SAMIL-V is the classifier fine-tuned using handheld imaging. W.Avg is the original cart-based classifier.

Figure 2

Confusion matrices at different prevalence of AS (2, 10, and 18%). Confusion matrices for the cart-based network and the fine-tuned SAMIL-V networks at varying disease prevalence. To obtain robust estimates of performance, we repeat all evaluations over five separate train/test splits of the 160 FoCUS scans in our dataset. Each split is drawn randomly and independently. The 2 and 10% prevalence are simulated by upsampling with replacement ‘no AS’ cases from the actual observed test set in each split. NN is neural network.