Misinterpreting cognitive change over multiple timepoints: When practice effects meet age-related decline

Abstract

Objective: Practice effects (PE) on cognitive testing have been shown to delay detection of impairment and impede our ability to assess change. When decline over time is expected, as with older adults or progressive diseases, failure to adequately address PEs may lead to inaccurate conclusions because PEs artificially boost scores while pathology- or age-related decline reduces scores. Unlike most methods, a participant-replacement approach can separate pathology- or age-related decline from PEs; however, this approach has only been used across two timepoints. More than two timepoints make it possible to determine if PEs level out after the first follow-up, but it is analytically challenging because individuals may not be assessed at every timepoint.

Method: We examined 1,190 older adults who were cognitively unimpaired (n = 809) or had mild cognitive impairment (MCI; n = 381). Participants completed six neuropsychological measures at three timepoints (baseline, 12-month, 24-month). We implemented a participant-replacement method using generalized estimating equations in comparisons of matched returnees and replacements to calculate PEs.

Results: Without accounting for PEs, cognitive function appeared to improve or stay the same. However, with the participant-replacement method, we observed significant PEs within both groups at all timepoints. PEs did not uniformly decrease across time; some-specifically on episodic memory measures-continued to increase beyond the first follow-up.

Conclusion: A replacement method of PE adjustment revealed significant PEs across two follow-ups. As expected in these older adults, accounting for PEs revealed cognitive decline. This, in turn, means earlier detection of cognitive deficits, including progression to MCI, and more accurate characterization of longitudinal change. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Figure 1:

Sample composition and replacement matching

Figure 2:. Expected cognitive scores among participants who were cognitively unimpaired at baseline

The Y-axis of each graph presents standardized scores for all 6 cognitive measures. The X-axis indicates the baseline (0), 12-month follow-up (1), and 24-month follow-up (2) visits. The blue line provides estimates from the practice effect-unadjusted generalized estimating equation model; the red line presents estimates from the practice effect-adjusted model. Dashed lines represent 95% confidence intervals for the practice effect-unadjusted models (blue line) and the practice effect-adjusted models (red lines). Lines with negative slopes indicate that participants’ scores are worsening as they age. All participants in these models were diagnosed as cognitively unimpaired at baseline. Trails A and Trials B were reverse scored to ease interpretation. Raw scores are not presented as differences in the scales of the cognitive tests made visualization unclear.

Figure 3:. Expected cognitive scores among participants diagnosed with mild cognitive impairment at baseline

The Y-axis of each graph presents standardized scores for all 6 cognitive measures. The X-axis indicates the baseline (0), 12-month follow-up (1) and 24-month follow-up (2). The blue line provides estimates from the practice effect-unadjusted generalized estimating equation model; the red line presents estimates from the practice effect-adjusted model. Dashed lines represent 95% confidence intervals for the practice effect-unadjusted models (blue line) and the practice effect-adjusted models (red lines). Lines with negative slopes indicate that participants’ scores are worsening as they age. All participants in these models were diagnosed with mild cognitive impairment at baseline. Trails A and Trials B were reverse scored to ease interpretation. Raw scores are not presented as differences in the scales of the cognitive tests made visualization unclear.