Response shift bias: a problem in evaluating nutrition training using self-report measures

Abstract

The Cooperative Extension Service has been a key partner in the design, implementation, and evaluation of school nutrition training. To evaluate the effectiveness of their training and the effects of response shift bias on outcomes using a self-report measure, 162 foodservice staff from eight rural schools participated in this food-handling behavior study. Nutrition staff were assigned to one of two treatment groups or to the control group. Two different evaluation designs (pretest/ post-test and then/post) were used. The then/post design asks participants to first report their behavior or understanding as a result of the training (post) and then to retrospectively report this behavior before the training. The then/post evaluation design provided more significant change data than did the traditional pretest/post-test design, indicating that a response shift occurred. Such differences in evaluation findings suggest that the educational benefit of such training may be underestimated when using the traditional pre/post evaluation design.