Unsupervised high-frequency smartphone-based cognitive assessments are reliable, valid, and feasible in older adults at risk for Alzheimer's disease

Abstract

Objective: Smartphones have the potential for capturing subtle changes in cognition that characterize preclinical Alzheimer's disease (AD) in older adults. The Ambulatory Research in Cognition (ARC) smartphone application is based on principles from ecological momentary assessment (EMA) and administers brief tests of associative memory, processing speed, and working memory up to 4 times per day over 7 consecutive days. ARC was designed to be administered unsupervised using participants' personal devices in their everyday environments.

Methods: We evaluated the reliability and validity of ARC in a sample of 268 cognitively normal older adults (ages 65-97 years) and 22 individuals with very mild dementia (ages 61-88 years). Participants completed at least one 7-day cycle of ARC testing and conventional cognitive assessments; most also completed cerebrospinal fluid, amyloid and tau positron emission tomography, and structural magnetic resonance imaging studies.

Results: First, ARC tasks were reliable as between-person reliability across the 7-day cycle and test-retest reliabilities at 6-month and 1-year follow-ups all exceeded 0.85. Second, ARC demonstrated construct validity as evidenced by correlations with conventional cognitive measures (r = 0.53 between composite scores). Third, ARC measures correlated with AD biomarker burden at baseline to a similar degree as conventional cognitive measures. Finally, the intensive 7-day cycle indicated that ARC was feasible (86.50% approached chose to enroll), well tolerated (80.42% adherence, 4.83% dropout), and was rated favorably by older adult participants.

Conclusions: Overall, the results suggest that ARC is reliable and valid and represents a feasible tool for assessing cognitive changes associated with the earliest stages of AD.

Keywords: digital biomarkers; ecological momentary assessment; mobile testing; preclinical Alzheimer’s disease.

Figure 1.. ARC Design and Cognitive Tasks

Note. Top demonstrates if a participant reported waking up at 7am and going to bed at 10pm, they would receive four test session notifications between 7am and 10pm, separated by at least two hours. The ARC cognitive tasks, Grids, Prices, and Symbols are displayed on the bottom.

Figure 2.. Between-Person Reliabilities for ARC Tasks

Note. Between-person reliabilities for each ARC cognitive task. Following Sliwinksi et al. (2018), a series of unconditional multilevel mixed models were fit to determine how many sessions would be required to obtain good reliability. Blue line indicates 0.85 reliability threshold.

Figure 3.. ARC Test-Retest Reliabilities at 6 Month (top) and 1 Year (bottom) Follow-Up

Figure 4.. ARC, Conventional, and AD Biomarker Correlations

Note. Correlations amongst ARC and conventional measures (raw scores) shown on the left (N = 282). Correlations of the ARC composite score (higher = worse) and global composite score (higher = better), and AD-related biomarkers are shown on the right (Ns = 146 for CSF measures, 212 for amyloid PET, 173 for tau PET, 175 for AD ROI cortical thickness, and 290 for hippocampal volume). Significant correlations (p < 0.05) are displayed with colored circles, non-significant correlations are blank. Because in-clinic and ARC measures have opposing directionality, the negative correlations amongst the conventional and ARC measures are in the hypothesized direction.

Figure 5.. Age, Technology Familiarity, and ARC Performance Correlations

Note. Of the 290 participants included in the present analyses, 220 completed the technology familarity survey (see Nicosia et al., 2021) which assessed the frequency with which participants perform smartphone-related tasks, how difficult participants find various technology-related tasks, and how well participants could recognize technology-related icons. Significant correlations (p < 0.05) are displayed with colored circles whereas non-significant relationships are blank.

Figure 6.. ARC User Experience Survey Results

Note. Of the 290 participants included in the present analyses, 228 completed the ARC user experience survey which assessed participants attitudes towards their experience with the ARC application after their first week using it.