Testing subject comprehension of utility questionnaires

Abstract

Utility questionnaires are often considered difficult for subjects to understand. Our study reports pilot testing of two subject comprehension tests to determine whether comprehension can be directly measured. Current health utilities were assessed using the standard gamble (SG), time trade-off (TTO), and visual analog scale. Subjects were randomized to one of two tests: (1) Logical consistency was tested by comparing rankings of two health states with an investigator-assigned a priori ranking; (2) Utility responses for two hypothetical respondents were presented; the subject was asked who had the better health. Thirty-one subjects completed the SG and TTO for two health states: being blind and wearing glasses. No subjects had inconsistent rankings. Post hoc analyses found that subjects reporting utilities below the first decile for the state, wearing glasses, had significantly lower current health utility than remaining subjects. Of the thirty subjects who evaluated the hypothetical respondents' utilities, five incorrectly judged the respondent with worse utility to have better health. Those subjects also reported current health utilities significantly lower than the remaining subjects. Our study findings suggest that a minority should be expected to have difficulty completing utility questionnaires. Comprehension checks may improve the reliability of utility data by enhancing training and by identifying subjects who may have misunderstood the utility questions.