Item response theory facilitated cocalibrating cognitive tests and reduced bias in estimated rates of decline

Abstract

Objective: To cocalibrate the Mini-Mental State Examination, the Modified Mini-Mental State, the Cognitive Abilities Screening Instrument, and the Community Screening Instrument for Dementia using item response theory (IRT) to compare screening cut points used to identify cases of dementia from different studies, to compare measurement properties of the tests, and to explore the implications of these measurement properties on longitudinal studies of cognitive functioning over time.

Study design and setting: We used cross-sectional data from three large (n>1000) community-based studies of cognitive functioning in the elderly. We used IRT to cocalibrate the scales and performed simulations of longitudinal studies.

Results: Screening cut points varied quite widely across studies. The four tests have curvilinear scaling and varied levels of measurement precision, with more measurement error at higher levels of cognitive functioning. In longitudinal simulations, IRT scores always performed better than standard scoring, whereas a strategy to account for varying measurement precision had mixed results.

Conclusion: Cocalibration allows direct comparison of cognitive functioning in studies using any of these four tests. Standard scoring appears to be a poor choice for analysis of longitudinal cognitive testing data. More research is needed into the implications of varying levels of measurement precision.

Appendix Figure 1

Running mean bias for three different scoring techniques in estimating rates of change over time.

Appendix Figure 1

Running mean bias for three different scoring techniques in estimating rates of change over time.

Appendix Figure 1

Running mean bias for three different scoring techniques in estimating rates of change over time.

Figure 1

Screening cut-points used in selected studies. Red = MMSE, blue = 3MS, black = CASI, green = CSI `D'. The x axis is the re-scaled IRT score, with a mean of 100 and a standard deviation of 15. Screening cutpoints were abstracted from the sources cited in the figure. Thus there is variability in screening cutpoints across studies that use the same instrument (identified in the figure by ink color). The unique contribution here is the ability to compare cutpoints across studies that used different tests, such as the MMSE cutpoint used by the Framingham study and the CASI cutpoint used by the ACT study. The cutpoints used in the different studies varied by nearly 4 standard deviations, a huge variation.

Figure 2

Test characteristic curves for tests of global cognitive functioning. Red = MMSE, blue = 3MS, black = CASI, green = CSI `D'. The x axis is the re-scaled IRT score, with a mean of 100 and a standard deviation of 15. Figure 2a. 3MS Figure 2b. CASI Figure 2c. CSI `D' Figure 2d. MMSE

Figure 3

Standard error of measurement for tests of global cognitive functioning. Red = MMSE, blue = 3MS, black = CASI, green = CSI `D'. The x axis is the re-scaled IRT score, with a mean of 100 and a standard deviation of 15.