Estimating Undercoverage Bias of Internet Users

Abstract

Introduction: In the last decade, response rates to the Behavioral Risk Factor Surveillance System (BRFSS) surveys have been declining. Attention has turned to the possibility of using web surveys to complement or replace BRFSS, but web surveys can introduce coverage bias as a result of excluding noninternet users. The objective of this study was to describe undercoverage bias of internet use.

Methods: We used data from 402,578 respondents who completed BRFSS questions in 2017 on internet use, self-reported health, current smoking, and binge drinking. We examined undercoverage bias of internet use by partitioning it into a product of 2 components: proportion of noninternet use and difference in the prevalences of interest (self-reported health, current smoking, and binge drinking) between internet users and noninternet users.

Results: Overall, the weighted proportion of noninternet use overall was 15.0%; the proportion increased with an increase in age and a decrease in education and, by race/ethnicity, was lowest among non-Hispanic white respondents. The overall relative bias was -19.2% for self-reported health, -4.0% for current cigarette smoking, and 8.4% for binge drinking. For all 3 variables of interest, we found large biases and relative biases in some demographic subgroups.

Conclusion: Undercoverage bias of internet use existed in the 3 studied variables. Both proportion of noninternet users and difference in prevalences of studied variables between internet users and noninternet users contributed to the bias to different degrees. These findings have implications on helping health-related behavioral risk factor surveys transition to more cost-effective survey modes than telephone only.