The myth of testing construct validity using factor analysis or correlations with normal or mixed clinical populations: lessons from memory assessment

Abstract

For nearly a century, the primary method employed by psychologists to define and test the validity of constructs evaluated by assessment instruments has been shared-variance techniques such as intervariable correlations or factor analysis with large normative or mixed clinical samples. To illustrate the shortcomings of this approach, we conducted (1) correlational analyses of immediate- and delayed-memory measures separately in normal participants and in homogeneous samples of patients with either Alzheimer's disease or Huntington's disease; and (2) factor analysis of immediate and delayed-recall and recognition measures in a large, homogeneous sample of patients with Alzheimer's disease. The findings revealed that cognitive measures that share variance in the intact brain-thereby giving the facade of assessing a unitary construct-can dissociate and contribute to unique variance in the damaged brain, but only if the pathology occurs in brain regions known to disrupt vital cognitive processes tapped by those measures. The results illustrate that shared-variance procedures applied to normal or mixed clinical populations can mask some of the most vital cognitive constructs, such as the classic distinction between short- and long-term memory. Implications of these findings for research and clinical practice are discussed.