No increase in response rate by adding a web response option to a postal population survey: a randomized trial

Abstract

Background: There is substantial interest in use of the Internet for surveys, but there have been few health-oriented, large, randomized trials of general population surveys on the Internet. It is unclear whether providing the option to respond via Internet increases the response rate, and to what degree the results will differ.

Objective: The aim of the study was to evaluate changes in response rate and outcomes in a postal respiratory health survey by adding an optional Web response alternative.

Methods: This was a randomized trial of a random sample of 4213 permanent residents of Norway, aged 20-40 years. Participants were randomized into a traditional survey arm, where they were asked to return the survey by mail, and an arm where they were also offered the option to respond via a Web form.

Results: A total of 1928/4213 subjects responded, a response rate of 45.8% across both arms. The total response rate was 44.8% (944/2105) in the postal plus optional Internet response arm and 46.7% (984/2108) in the usual postal survey arm, with no statistically significant difference between the randomized groups (P = .24). In the optional Internet arm, 8.3% (175/2105) of the sample responded using the Internet and 36.5% (769/2105) responded by post. Thus, Internet response was chosen by 18.5% (175/944) of those who replied in the optional Internet arm. In the multivariate analysis, Internet response was associated with being male, frequency and type of Internet access (home users more likely to respond by Internet than work users), and smoking habit, with current smokers being more likely to be Internet responders. 57% preferred postal response (1102/1928), 38% preferred Internet response (733/1928), and 3% preferred telephone interview (54/1928), with no difference between randomization arms (P = .56). But among those who indicated that they preferred the Internet response and who were randomized to the optional Internet arm, only 47% actually chose the Internet response. Asthma prevalence was higher among participants choosing the Internet response mode (16.7% vs 12.4%).

Conclusions: We failed to increase survey response rates by adding an optional Internet response. Asthma diagnosis was higher in the Internet response group, suggesting nonresponse bias. Method comparison studies should be carried out before Internet studies are accepted in new populations or new subject matters.