The importance of assessing clinical phenomena in Mechanical Turk research

Abstract

Amazon.com's Mechanical Turk (MTurk) website provides a data collection platform with quick and inexpensive access to diverse samples. Numerous reports have lauded MTurk as capturing high-quality data with an epidemiological sample that is more representative of the U.S. population than traditional in-person convenience samples (e.g., undergraduate subject pools). This benefit, in combination with the ease and low-cost of data collection, has led to a remarkable increase in studies using MTurk to investigate phenomena across a wide range of psychological disciplines. Multiple reports have now examined the demographic characteristics of MTurk samples. One key gap remains, however, in that relatively little is known about individual differences in clinical symptoms among MTurk participants. This paper discusses the importance of assessing clinical phenomena in MTurk samples and supports its assertions through an empirical investigation of a large sample (N = 1,098) of MTurk participants. Results revealed that MTurk participants endorse clinical symptoms to a substantially greater degree than traditional nonclinical samples. This distinction was most striking for depression and social anxiety symptoms, which were endorsed at levels comparable with individuals with clinically diagnosed mood and anxiety symptoms. Participants' symptoms of physiological anxiety, hoarding, and eating pathology fell within the subclinical range. Overall, the number of individuals exceeding validated clinical cutoffs was between 3 and 19 times the estimated 12-month prevalence rates. Based on the current findings, it is argued that MTurk participants differ from the general population in meaningful ways, and researchers should consider this when referring to this sample as truly representative. (PsycINFO Database Record