The unreliable change of reliable change indices

Abstract

The classic method for assessment of reliable change, in 1991 re-introduced as Jacobson's RC, can be characterized as a confidence interval method. In recent years, several RC indices have been proposed using Kelley's (1947) (Kelley, T. L. (1947). Fundamentals of statistics. Cambridge: Harvard University Press) formula for estimating true change. In these proposals, interval estimation and confidence intervals are mixed up, which leads to unjustified probability statements. When Kelley's estimate is correctly expanded into a normal distributed statistic, the classic approach reveals itself as a large sample approximation of a properly constructed RCI based on Kelley's formula. Researchers should continue using the classic approach for the determination of reliable change.