Robust cognitive change

Abstract

Two major challenges facing researchers interested in cognitive change are that measures of change are often not very reliable, and they may reflect effects of prior test experience in addition to the factors of primary interest. One approach to dealing with these problems is to obtain multiple measures of change on parallel versions of the same tests in a measurement burst design. A total of 783 adults performed three parallel versions of cognitive tests on two occasions separated by an average of 2.6 years. Performance increased substantially across the three sessions within each occasion, and for all but vocabulary ability these within-occasion improvements were considerably larger than the between-occasion changes. Reliabilities of the changes in composite scores were low, but averages of the three changes had larger, albeit still quite modest, reliabilities. In some cognitive abilities individual differences were evident in the relation of prior test experience and the magnitude of longitudinal change. Although multiple assessments are more time consuming than traditional measurement procedures, the resulting estimates of change are more robust than those from conventional methods, and also allow the influence of practice on change to be systematically investigated.

Fig. 1

Means and standard errors of average change in composite scores for five cognitive abilities across the three sessions (solid symbols and lines), and of the proportion of individuals with consistent gain or consistent loss across the three changes.

Fig. 2

Means and standard errors of composite scores on the T1 and T2 occasions for each of the three sessions.