The impact of overtly listing eligibility requirements on MTurk: An investigation involving organ donation, recruitment scripts, and feelings of elevation

Abstract

Rationale: Investigations conducted through Amazon's Mechanical Turk (MTurk) sometimes explicitly note eligibility requirements when recruiting participants; however, the impact of this practice on data integrity is relatively unexplored within the MTurk context.

Objective: Contextualized in the organ donor registration domain, the current study assessed whether overtly listing eligibility requirements impairs the accuracy of data collected on MTurk.

Methods: On day 1, the first and third round of data collection did not list eligibility requirements; the second and fourth round overtly listed a qualification requirement: status as a non-registered organ donor. On day 2, the approach was identical, except the order was reversed-the first and third round overtly listed the study qualifications, while the second and fourth did not. These procedures provided eight different waves of data. In addition, all participants were randomly assigned to read an elevating (i.e., morally inspiring) story or a story not intended to elicit any emotion. Regardless of recruitment approach, only participants who were not registered as donors were included in the analysis.

Results: Results indicated that the recruitment script that explicitly requested non-registered donors resulted in the collection of participants with higher mean intentions scores than the script that did not overtly list the eligibility requirements. Further, even though the elevation induction increased intentions to register as a donor, there was not a significant interaction between recruitment approach and the influence of the elevation manipulation on registration intentions.

Conclusion: Explicitly listing eligibility requirements can influence the accuracy of estimates derived from data collected through MTurk.

Keywords: Elevation; Eligibility requirements; MTurk; Methods; Organ donation; Organ donor registration; Recruitment.