Methods to detect response shift in quality of life data: a convergent validity study

Abstract

When measuring changes in quality of life (QL) with a pretest-posttest design, response shift can affect results. We investigated the convergent validity of three approaches to detect response shift. (1) In the thentest approach, response shift is measured using a retrospective judgment of pretest QL-levels (thentest). (2) In the anchor-recalibration approach response shift is measured, assessing shifts in patients' individual definitions of the scale-anchors (worst and best imaginable QL) over time. (3) In the Structural Equation Modeling (SEM) approach response shift is indicated by mathematically defined changes in factor solutions and variance-covariance matrices over time. Prior to and three months after invasive surgery, 170 cancer patients completed the SF-36, the Multidimensional Fatigue Inventory (as pre-, post-, and thentest), and the anchor-recalibration task (as pre-, and posttest). Results showed agreement between the thentest and SEM approach on the absence (6 scales) and presence (2 scales) of response shift in 8 of the 9 scales. For the ninth scale both methods detected response shift, but in opposite directions. Possible explanations for this discrepancy are discussed. The anchor-recalibration task agreed with the other approaches on only the absence of response shift in 4 of the 7 scales. The convergent results of thentest and SEM support their validity, especially because they use statistically independent operationalizations of response shift. In this study, recall bias did not invalidate thentest results.