Stricker Learning Span criterion validity: a remote self-administered multi-device compatible digital word list memory measure shows similar ability to differentiate amyloid and tau PET-defined biomarker groups as in-person Auditory Verbal Learning Test

Abstract

Objective: The Stricker Learning Span (SLS) is a computer-adaptive digital word list memory test specifically designed for remote assessment and self-administration on a web-based multi-device platform (Mayo Test Drive). We aimed to establish criterion validity of the SLS by comparing its ability to differentiate biomarker-defined groups to the person-administered Rey's Auditory Verbal Learning Test (AVLT).

Method: Participants (N = 353; mean age = 71, SD = 11; 93% cognitively unimpaired [CU]) completed the AVLT during an in-person visit, the SLS remotely (within 3 months) and had brain amyloid and tau PET scans available (within 3 years). Overlapping groups were formed for 1) those on the Alzheimer's disease (AD) continuum (amyloid PET positive, A+, n = 125) or not (A-, n = 228), and those with biological AD (amyloid and tau PET positive, A+T+, n = 55) vs no evidence of AD pathology (A-T-, n = 195). Analyses were repeated among CU participants only.

Results: The SLS and AVLT showed similar ability to differentiate biomarker-defined groups when comparing AUROCs (p's > .05). In logistic regression models, SLS contributed significantly to predicting biomarker group beyond age, education, and sex, including when limited to CU participants. Medium (A- vs A+) to large (A-T- vs A+T+) unadjusted effect sizes were observed for both SLS and AVLT. Learning and delay variables were similar in terms of ability to separate biomarker groups.

Conclusions: Remotely administered SLS performed similarly to in-person-administered AVLT in its ability to separate biomarker-defined groups, providing evidence of criterion validity. Results suggest the SLS may be sensitive to detecting subtle objective cognitive decline in preclinical AD.

Keywords: Alzheimer’s disease; aging; cognition; mobile health; neuropsychological tests; neuropsychology; smartphone; telemedicine.

Figure 1.

Stricker Learning Span methods. Figure used with permission of Mayo Foundation for Medical Education and Research; all rights reserved.

Figure 2.

Stricker Learning Span (SLS) computer adaptive testing approach provides an expanded ceiling and floor of items presented relative to traditional word list memory tests. High performers are exposed to an increasing number of words across trials as shown in blue. Low performers are shown a decreasing number of words across each trial as shown in orange. Figure used with permission of Mayo Foundation for Medical Education and Research; all rights reserved.

Figure 3.

Area Under the Receiving Operating Curve (AUROC, 95% CI) values for remotely-administered SLS sum of trials and in-person administered AVLT sum of trials. Note. All models significantly differentiate biomarker groups better than chance (no AUROC confidence intervals include 0.5). Stricker Learning Span (SLS) sum of trials = 1–5 total correct + delay. Auditory Verbal Learning Test (AVLT) sum of trials = 1–5 total + Trial 6 + 30-minute delay recall. Unadj. = unadjusted models. Adj. = model adjusts for age, education and sex. Figure used with permission of Mayo Foundation for Medical Education and Research; all rights reserved.

Figure 4.

Hedge’s g effect sizes with 95% confidence intervals to show the magnitude of group differences for remotely administered Stricker Learning Span (SLS, red shades) and in-person Auditory Verbal Learning Test (AVLT) measures (blue shades) across biomarker groups (A− vs A+ top; A−T− vs A+T+ bottom) in all participants (left) and Cognitively Unimpaired participants (right). Note. Groups do not significantly differ when the CI includes 0 (dashed line). Unadj. = unadjusted models. Adj. = model adjusts for age, education and sex. Remote SLS Sum of Trials = SLS trials 1–5 correct + delay correct. In-person AVLT Sum of Trials = AVLT 1–5 correct + Trial 6 (short-delay) correct + 30-minute recall correct). See Tables 4 and 5 for direction of effect sizes and confidence intervals. See Supplemental Table 2 for numeric results. Figure used with permission of Mayo Foundation for Medical Education and Research; all rights reserved.

Figure 5.

Hedge’s g unadjusted effect sizes comparing ability of learning and delay trials to differentiate individuals without AD biomarkers from those with biological AD (A−T− vs A+T+). Note. Stricker Learning Span (SLS) Trials 1–5 = trials 1–5 total correct; Auditory Verbal Learning Test (AVLT) Trials 1–5 = 1–5 total correct; AVLT Delay =30-minute delayed recall. See Supplemental Table 2 for numeric results. Figure used with permission of Mayo Foundation for Medical Education and Research; all rights reserved.

Figure 6.

Stricker Learning Span (SLS, left panel) and Auditory Verbal Learning Test (AVLT, right panel) learning slopes and delayed memory performances across all participants without AD biomarkers (A−T−, blue) and all participants with biological AD (A+T+, red). Note. Sum of trials is the primary variable for each test; these figures show the data comprising sum of trials for each measure (sum of trials = total correct items for all trials displayed in each respective figure). SLS items presented differ by trial based on computer adaptive testing rules, thus the highest possible number of words presented across trials vary (see Figure 1 for ceiling and floor values for Trials 1–5; the delay trial can range from 8–23). The SLS uses 4-choice recognition to test item memory. In contrast, 15 words are presented each time for AVLT trials (constant range of 0–15) and item memory is tested with free recall. See Supplemental Table 2 for numeric results. AVLT Trial 6 = short delay. AVLT delay = 30-minute delay. Figure used with permission of Mayo Foundation for Medical Education and Research; all rights reserved.