Test-retest effects in treatment studies of reading disability: the devil is in the detail

Abstract

Reynolds and Nicolson (Dyslexia, 2007; 13: 78-96) claim to show that the 'dyslexia dyspraxia attention-deficit treatment' (DDAT) benefits children with reading difficulties. However, Rack, Snowling, Hulme, and Gibbs (Dyslexia, 2007; 13: 97-104) argue that because this study did not include an untrained control group then 'all that needs to be postulated to explain the results reported is that children improve their scores on the DST screening tests simply as a result of repeated testing on the same activities' (p. 102). How likely is it that the linguistic gains reported by Reynolds and Nicolson (Dyslexia, 2007; 13: 78-96) are due to test-retest effects? The results of previous exercise- and auditory-based treatment studies that included an untrained control group suggest that test-retest effects explain gains on around 50% of real-word reading tests, 33% of phonological recoding tests, 33% of phonological awareness tests, 17-25% of spoken language tests, and 15% of spelling tests. In addition, longer periods of time between test and retest sessions are associated with test-retest effects on measures of reading but not spoken language. These findings suggest that two of the four linguistic gains reported by Reynolds and Nicolson (Dyslexia, 2007; 13: 78-96) are due to test-retest effects (phonemic segmentation and working memory). The remaining two tests are measures of spoken language and not reading. Hence, the data reported by Reynolds and Nicolson (Dyslexia, 2007; 13: 78-96) are not sufficient to support DDAT as an effective treatment for children with reading difficulties.