Digital detection of Alzheimer's disease using smiles and conversations with a chatbot

Abstract

In super-aged societies, dementia has become a critical issue, underscoring the urgent need for tools to assess cognitive status effectively in various sectors, including financial and business settings. Facial and speech features have been tried as cost-effective biomarkers of dementia including Alzheimer's disease (AD). We aimed to establish an easy, automatic, and extensive screening tool for AD using a chatbot and artificial intelligence. Smile images and visual and auditory data of natural conversations with a chatbot from 99 healthy controls (HCs) and 93 individuals with AD or mild cognitive impairment due to AD (PwA) were analyzed using machine learning. A subset of 8 facial and 21 sound features successfully distinguished PwA from HCs, with a high area under the receiver operating characteristic curve of 0.94 ± 0.05. Another subset of 8 facial and 20 sound features predicted the cognitive test scores, with a mean absolute error as low as 5.78 ± 0.08. These results were superior to those obtained from face or auditory data alone or from conventional image depiction tasks. Thus, by combining spontaneous sound and facial data obtained through conversations with a chatbot, the proposed model can be put to practical use in real-life scenarios.

Fig. 1

Composition of sections considered when extracting of facial features from the smile test.

Fig. 2

Receiver operating characteristic (ROC) curve of the machine learning algorithms. (A) Random forest algorithm using facial and sound feature sets (No. 3 in Table 2). (B) Logistic regression algorithm using facial and sound feature sets (No. 6 in Table 2). These algorithms had the best classification performance. The horizontal axis is the false-positive rate, and the vertical axis is the true-positive rate. The light blue lines are the ROC curves of the cross-validation, and the dark blue lines are the averaged ROC curves. The green dashed line indicates the performance of a random classifier.

Fig. 3

Distribution of the mobile application cognitive test score prediction using different algorithms. (A) light gradient boosting machine algorithm with facial and sound feature sets (No. 12 in Table 3). (B) Ridge regression algorithm with facial and sound feature sets (No. 6 in Table 3). The horizontal axis (observation) is the true application cognitive test scores, and the vertical axis (prediction) is the predicted application cognitive test scores. The blue lines show the correlation between the true and predicted values, and the green dashed lines indicate the score distribution when the correlation coefficient = 1.