Discriminant construct validity of ImPACT™: a companion study

Abstract

In a previous analysis of ImPACT scores relative to traditional neuropsychological tests (NP) and experimental tasks (Maerlender et al., 2010 ) we demonstrated convergent construct validity for the primary ImPACT test-score composites. A complete analysis of discriminant validity was not undertaken at that time. Here, test scores from the 54 collegiate football and hockey players were re-analyzed to specifically address the discriminant validity of the ImPACT™ composite scores using a multiply operationalized correlation matrix of multi-trait multi-method data. In the method used here, discriminant validity is determined by obtaining non-significant correlations between a target score when correlated with the average of the other trait measurements (multiply-operationalized multi-trait-mono-method analysis). Results showed that the ImPACT™ Verbal Memory (p = .044), Visual Memory (p = .006), and Visual Motor Speed (p = .000) scores were highly correlated with composites of the other scores, while the Reaction Time composite demonstrated adequate discriminant validity (p = .145). In comparison all of the NP composites showed good discrimination (all p-values >.05, except for Reaction Time p = .05). Thus the apparent lack of discriminability between three of four composite scores in this sample raises questions about using ImPACT™ composite scores to support specific construct-oriented interpretations. Taken together, the discriminant and convergent construct validity properties of ImPACT™ indicate construct sensitivity, but limited construct specificity.

Figure 1

Hetero-Trait Mono-Method Multiple Operationalization