The validity of qualtrics panel data for research on video gaming and gaming disorder

Abstract

Crowdsourced samples are increasing in popularity, particularly within psychological and addictive behaviors research. The trend has resulted in significant interest in the use of panel samples for the examination of behavioral and substance addictions. One newer panel platform, Qualtrics, has seen an increase in usage in recent years despite lack of research examining the validity of Qualtrics-produced data for addictive behaviors. The goal of the present study was to evaluate the validity of Qualtrics-obtained data for the most recently classified behavioral addiction-video gaming. The evaluation compared a Qualtrics-recruited video gamers sample (n = 586) to traditional community (n = 108) and student (n = 217) samples on demographics and key outcomes relevant to gaming disorder researchers (average playtime, frequency of gaming, and gaming disorder risk scores) to evaluate the generalizability of Qualtrics panel data. The results revealed that Qualtrics samples were comparable to a traditionally recruited community sample, but different from a student sample on gaming frequency (p < .001) and risk for gaming disorder (p < .001). The Qualtrics sample also had longer durations of average gaming time relative to the student sample (p = .01), with some differences in demographics between the all three sources of recruitment. The findings suggest that Qualtrics may provide a suitable method of convenience panel recruitment, generalizable to the broader North American community, for research examining video gaming behaviors and gaming disorder. (PsycInfo Database Record (c) 2022 APA, all rights reserved).