A remote digital memory composite to detect cognitive impairment in memory clinic samples in unsupervised settings using mobile devices

Abstract

Remote monitoring of cognition holds the promise to facilitate case-finding in clinical care and the individual detection of cognitive impairment in clinical and research settings. In the context of Alzheimer's disease, this is particularly relevant for patients who seek medical advice due to memory problems. Here, we develop a remote digital memory composite (RDMC) score from an unsupervised remote cognitive assessment battery focused on episodic memory and long-term recall and assess its construct validity, retest reliability, and diagnostic accuracy when predicting MCI-grade impairment in a memory clinic sample and healthy controls. A total of 199 participants were recruited from three cohorts and included as healthy controls (n = 97), individuals with subjective cognitive decline (n = 59), or patients with mild cognitive impairment (n = 43). Participants performed cognitive assessments in a fully remote and unsupervised setting via a smartphone app. The derived RDMC score is significantly correlated with the PACC5 score across participants and demonstrates good retest reliability. Diagnostic accuracy for discriminating memory impairment from no impairment is high (cross-validated AUC = 0.83, 95% CI [0.66, 0.99]) with a sensitivity of 0.82 and a specificity of 0.72. Thus, unsupervised remote cognitive assessments implemented in the neotiv digital platform show good discrimination between cognitively impaired and unimpaired individuals, further demonstrating that it is feasible to complement the neuropsychological assessment of episodic memory with unsupervised and remote assessments on mobile devices. This contributes to recent efforts to implement remote assessment of episodic memory for case-finding and monitoring in large research studies and clinical care.

Fig. 1. Memory tests constituting the RDMC score.

a Mnemonic discrimination test for objects and scenes (MDT-OS). b Object-in-room-recall (ORR) test. c Complex scene recognition test (CSR).

Fig. 2. Timeline of the study protocol.

Participants enlisted in a 12-month remote memory assessment study. During the initial session, subjects gave consent to participate in the study and installed the mobile app. For each session, they completed a short tutorial, followed by phase 1 of their respective task: encoding for ORR and CSR, and 1 back task for MDT-OS. Following the delay period, participants were notified that the next phase was available and could perform it immediately or postpone it if testing was inconvenient. Phase 2 consisted of retrieval for ORR and CSR and 2-back task for MDT-OS. It was followed by ratings regarding concentration, subjective performance, and distraction during the task. While most participants were notified regarding the next available test session every other week (continuous design), a subgroup within the WRAP cohort performed all three paradigms in a burst on four consecutive days every eight weeks (burst design).

Fig. 3. Construct validity of the composite score.

Correlation between the RDMC (one session of each paradigm) and the closest-in-time a DELCODE PACC5 assessment as well as b–f all individual PACC5 elements. HC Healthy Controls, SCD subjective cognitive decline, MCI mild cognitive impairment, FCSRT free and cued selective reminding test, MMSE mini-mental state exam.

Fig. 4. Diagnostic accuracy across Single Session and Three Session scenarios.

a, c Receiver operating characteristic (ROC) curve showing diagnostic accuracy for the detection of MCI-grade cognitive impairment based on the RDMC. b, d Scatter plot showing the optimal RDMC cut-off with CU and CI in different colors—optimal cut-offs at −0.35 and −0.41 indicated by the dashed gray line, dots to the left are classified as cognitively impaired, and dots to the right as cognitively unimpaired. Information is shown for the single-session scenario (i.e., one session of each paradigm) and the three-session scenario.