New technology in quality of life research: are all computer-assisted approaches created equal?

Abstract

Objective: To compare alternate form reliability for two previously validated questionnaires using traditional and computer-assisted interfaces (CASI).

Methods: The Rhinoconjunctivitis Quality of Life Questionnaire (RQLQ) and the allergy-specific Work Productivity and Activity Impairment (WPAI) instrument were combined and formatted for paper-and-pencil (P), computer touch screen (TS), or web-based (WB) administration. Employees at a large midwestern hospital in the US each completed two of three possible formats assigned randomly and in random order. Those completing one of three possible format pairs (P-TS, P-WB, and TS-WB) comprised three study groups. Item response consistency was assessed using Spearman correlations; individuals' test-retest scores were compared using paired t-tests with mean test-retest differences compared between groups using ANOVA.

Results: Seventy five volunteers completed 150 questionnaires. Item response consistency varied by format pair with correlations ranging from 0.566 to 0.973. Although no differences were observed in paired RQLQ responses, participants completing the WB form reported generally higher WPAI scores compared with those from either the paper (p = 0.07) or TS (p = 0.001) format. ANOVA of mean RQLQ and WPAI scores demonstrated no between-group differences.

Conclusions: Use of CASI, an increasingly popular method in survey research, may have important effects on instrument reliability. Studies examining this phenomenon more closely are needed to guide future use of CASI in this setting.