Early detection of cognitive decline with deep learning and graph-based modeling

Abstract

In today's world, increasing stress and depression significantly impact cognitive well-being, making early detection of cognitive impairment essential for timely intervention. This work introduces a Multimodal Fusion Cognitive Assessment Framework that leverages advanced deep learning and graph intelligence to enhance early identification accuracy. Traditional tools like the Montreal Cognitive Assessment (MOCA) are limited in adaptability, prompting the need for a more dynamic, data-driven approach. The framework is validated using datasets involving cognitive tests, voice samples, and physiological signals. It enables a scalable, personalized, and adaptive cognitive assessment system that improves early detection and supports targeted intervention strategies. By integrating deep learning and information fusion, this approach addresses the complexity of cognitive health in a modern context.•This paper introduces Multimodal Deep Learning Integration, incorporating MOCA scores, behavioral data, speech signals, and physiological parameters using GAT, TAT, and CNN-LSTM models to capture diverse cognitive indicators.•The proposed model achieves superior performance through Information Fusion via Heterogeneous GNNs, effectively merging cross-domain data to enable holistic cognitive state assessment via inter-modality learning.•This paper applies Reinforcement Learning (RL) to personalize user interactions based on real-time cognitive and stress cues, reducing cognitive overload and enhancing engagement.

Keywords: Cognitive Anomaly Detection and Adaptive Chatbot Response; Cognitive assessment; Deep learning; Graph neural networks; Multimodal fusion.

Graphical abstract

Fig. 1

Cognitive skill impairment levels and effects.

Fig. 2

Proposed chatbot process model architecture.

Fig. 3

Workflow of the proposed chatbot system.

Fig. 4

Models integrated performance.

Fig. 5

Model’s overall result analysis.

Fig. 6

Multimodal cognitive score integration on combined dataset.