Artificial Intelligence-Enabled End-To-End Detection and Assessment of Alzheimer's Disease Using Voice

Abstract

There is currently no simple, widely available screening method for Alzheimer's disease (AD), partly because the diagnosis of AD is complex and typically involves expensive and sometimes invasive tests not commonly available outside highly specialized clinical settings. Here, we developed an artificial intelligence (AI)-powered end-to-end system to detect AD and predict its severity directly from voice recordings. At the core of our system is the pre-trained data2vec model, the first high-performance self-supervised algorithm that works for speech, vision, and text. Our model was internally evaluated on the ADReSSo (Alzheimer's Dementia Recognition through Spontaneous Speech only) dataset containing voice recordings of subjects describing the Cookie Theft picture, and externally validated on a test dataset from DementiaBank. The AI model can detect AD with average area under the curve (AUC) of 0.846 and 0.835 on held-out and external test set, respectively. The model was well-calibrated (Hosmer-Lemeshow goodness-of-fit p-value = 0.9616). Moreover, the model can reliably predict the subject's cognitive testing score solely based on raw voice recordings. Our study demonstrates the feasibility of using the AI-powered end-to-end model for early AD diagnosis and severity prediction directly based on voice, showing its potential for screening Alzheimer's disease in a community setting.

Keywords: Alzheimer’s disease; data2vec; dementia; end-to-end; large language models; speech and language.

Figure 1

Overview of the AI-powered end-to-end model for AD diagnosis and disease severity prediction from voice. The system takes as input the voice recording from a subject that is converted into the embedding by data2vec, which is then fed into a neural network to determine the AD status and assess the severity of their AD in accordance with MMSE score.

Figure 2

ROC curves for detecting AD using the data2vec from the ADReSSo unseen test set with random forest (RF), support vector classifier (SVC), logistic regression (LR), and neural network (NN) models. The mean AUC and the 95% confidence interval are reported for all four models.

Figure 3

Model Performance for AD detection task using the data2vec embeddings from the ADReSSo unseen test data. The error bar in each metric denotes the 95% confidence interval.

Figure 4

Reliability diagrams of the calibrated and uncalibrated neural network (NN) using isotonic regression.

Figure 5

ROC curves and the corresponding AUC scores of the neural network (NN) classifier on the external DementiaBank Pitt dataset and internal unseen ADReSSo test set. 95% confidence interval is also reported for the AUC score.

Figure 6

Boxplots of the mean absolute error results for embeddings from data2vec and wav2vec2 and acoustic feature set using the AD severity predictor based on the 10-fold cross-validation. A Kruskal–Wallis H-test showed that there is a statistically significant difference between the groups (H = 6.9083, p = 0.0316). A post hoc Dunn’s test further showed that data2vec is statistically significantly different at p = 0.0103 (*).