The potential synergy between cognitive models and modern psychometric models

Abstract

Analyses of cognitive aspects of survey methodology (CASM) and psychometric analysis are two methods that are able to complement each other. We use concrete examples to illustrate how psychometric analyses can test hypotheses from CASM. The psychometrics framework recognizes that survey responses are affected by other factors than the concept being assessed, for example by cognitive factors and processes. Such factors are subsumed under the concept of measurement error. Possible sources of measurement error can be tested, e.g. by randomized experiments. A standard way to reduce measurement error is to ask several questions about the same concept and combine the answers into a multi-item scale that is more precise than the individual items. Techniques like structural equation models use the item correlations to assess the magnitude of measurement error and to test the assumptions behind the multi-item scale, e.g. the effect of common response choices and item time frames. A central problem in modern psychometrics is how to model the mapping of the continuous latent variable onto the item response choice categories. This is achieved by threshold models (e.g. item response models and structural equation models for categorical data). These models can, for example, analyze the impact of mode of administration, test whether the items function in the same way for all people (measurement invariance/differential item functioning) and examine the consistency of responses from any single person. Such analyses provide new possibilities for combining psychometrics and cognitive methods.