Enhanced detection of severe aortic stenosis via artificial intelligence: a clinical cohort study

Abstract

Objective: We developed an artificial intelligence decision support algorithm (AI-DSA) that uses routine echocardiographic measurements to identify severe aortic stenosis (AS) phenotypes associated with high mortality.

Methods: 631 824 individuals with 1.08 million echocardiograms were randomly spilt into two groups. Data from 442 276 individuals (70%) entered a Mixture Density Network (MDN) model to train an AI-DSA to predict an aortic valve area <1 cm², excluding all left ventricular outflow tract velocity or dimension measurements and then using the remainder of echocardiographic measurement data. The optimal probability threshold for severe AS detection was identified at the f1 score probability of 0.235. An automated feature also ensured detection of guideline-defined severe AS. The AI-DSA's performance was independently evaluated in 184 301 (30%) individuals.

Results: The area under receiver operating characteristic curve for the AI-DSA to detect severe AS was 0.986 (95% CI 0.985 to 0.987) with 4622/88 199 (5.2%) individuals (79.0±11.9 years, 52.4% women) categorised as 'high-probability' severe AS. Of these, 3566 (77.2%) met guideline-defined severe AS. Compared with the AI-derived low-probability AS group (19.2% mortality), the age-adjusted and sex-adjusted OR for actual 5-year mortality was 2.41 (95% CI 2.13 to 2.73) in the high probability AS group (67.9% mortality)-5-year mortality being slightly higher in those with guideline-defined severe AS (69.1% vs 64.4%; age-adjusted and sex-adjusted OR 1.26 (95% CI 1.04 to 1.53), p=0.021).

Conclusions: An AI-DSA can identify the echocardiographic measurement characteristics of AS associated with poor survival (with not all cases guideline defined). Deployment of this tool in routine clinical practice could improve expedited identification of severe AS cases and more timely referral for therapy.

Keywords: Aortic Valve Stenosis; Echocardiography; Translational Medical Research.

Figure 1

Flow chart of the training, evaluation, and testing of the AI-DSA to detect severe forms of AS. This schema shows the distribution of cases/investigations from the NEDA cohort used to train and evaluate the AI-DSA (investigation-based, no mortality data) and then test/assess its performance in accurately detecting the severe form of aortic stenosis associated with high 5-year mortality (individual-based). The highest F1 score was chosen as the probability at the peak of the precision/recall relationship, corresponding to a probability of 0.235. The moderate-to-severe aortic stenosis group corresponds to a probability output of>0.0625 (98.25 to 98.5 percentile of probability spectrum below the f1-derived threshold). AI-DSA, Artificial Intelligence Decision Support Algorithm; AS, aortic stenosis; AV, aortic valve; BSA, body surface area, F/U, follow-up; LVEF, left ventricular ejection fraction; LVOT, left ventricular outflow tract; NEDA, National Echo Database of Australia; pct, percentile.

Figure 2

Performance of the model to detect severe AS. This graph shows the performance of the model underpinning the AI-DSA to identify an aortic valve area of<1.0 cm². AI-DSA, Artificial Intelligence Decision Support Algorithm; AS, aortic stenosis; FPR, false positive rate; LVEF, left ventricular ejection fraction; NEDA, National Echo Database of Australia; TPR, true positive rate.

Figure 3

Precision-recall performance of the model to detect severe AS. This graph shows the precision values (true positives/(true positives+false negatives)) on the y-axis and recall values (true positives/(true positives+false positives) on the x-axis derived from the AI-DSA output. AI-DSA, Artificial Intelligence Decision Support Algorithm; AS, aortic stenosis; LVEF, left ventricular ejection fraction; NEDA, National Echo Database of Australia.

Figure 4

Probability threshold of the model to detect severe AS. This graph shows the plots used to determine the F1-derived threshold based on the average of precision and recall of the AI-DSA to detect severe aortic stenosis (main red dotted line) overall (F1-derived probability threshold 0.235) and in those with a left ventricular ejection fraction <50% and <30%. It also shows (short black dotted line)—the 0.0625 probability threshold for identifying ‘moderate aortic stenosis’ group. AI-DSA, Artificial Intelligence Decision Support Algorithm; AS, aortic stenosis; LVEF, left ventricular ejection fraction; NEDA, National Echo Database of Australia.

Figure 5

Actual 5-year all-cause mortality according to three main outputs from the AI-DSA. This graph shows the 5- year actual mortality curves (all-cause and with no censoring of cases) for the three main output groups from the AI-DSA—with low probability individuals being the reference group). The ORs for age and sex were 1.08 (95% CI 1.08 to 1.08 per annum) and 1.50 (95% CI 1.46 to 1.55 for men vs women). AI-DSA, Artificial Intelligence Decision Support Algorithm.

Figure 6

Actual 5-year all-cause mortality in the two severe AS groups (AI-DSA identified vs guidelines). This graph shows the 5-year actual mortality curves (all-cause and with no censoring of cases) for the two output groups identified by the AI-DSA as severe aortic stenosis—according to clinical guideline criteria (black line) or otherwise (red line—reference group). The ORs for age and sex were 1.09 (95% CI 1.08 to 1.10 per annum) and 1.28 (95% CI 1.08 to 1.53 for men vs women). AI-DSA, Artificial Intelligence Decision Support Algorithm; AS, aortic stenosis.