Testing certain and uncertain incentives on study retention in a longitudinal social media survey among young adults: An embedded recruitment trial

Abstract

Introduction: Incentives can be effective in survey research but evidence is limited on how incentive type impacts survey retention in longitudinal social media-based surveys. This study examined how certain and uncertain incentives affect study retention among US young adults recruited online and whether incentive effects vary by sociodemographic factors.

Methods: Participants were randomized in a 1:1:1 ratio to a three-arm parallel trial (n = 1615) with (1) a lottery for a $200 gift card (uncertain), (2) a cash equivalent (CE) of a $5 gift card per survey (certain); or (3) a combination of both options (combined), and were surveyed at baseline, 30 days, and 60 days. This study focused on survey retention at 30 days (among baseline completers, n = 1491) and 60 days (among 30-day completers, n = 1018). Participants were not blinded to their condition but were blinded to other conditions and researchers were blinded until data collection was complete. Logistic regressions examined survey retention as a function of incentive condition and sociodemographics, with additional analyses of interaction effects. We report average marginal effects (AMEs) with significance defined as p < 0.05.

Results: The certain CE was effective for survey retention versus the lottery at 30-day follow-up only (43.8% [lottery] vs. 77.7% [CE], AME: 0.346, p < 0.000); there were no differences between CE versus lottery at 60-day follow-up (76.1% [lottery] and 81.3% [CE], AME: 0.054, p = 0.192). The combined incentive demonstrated significantly higher retention at both follow-ups versus the lottery but no significant advantage over the CE. Incentive effectiveness showed minimal variation across sociodemographic factors.

Discussion: This study is among the few to experimentally test incentives for retention in online social-media based research. A certain CE was most effective for short-term web survey retention among young adults compared with a lottery. Findings suggest that small guaranteed rewards may better motivate study retention than uncertain larger amounts.

Keywords: Incentives; behavioral economics; e-cigarettes; prospect theory; retention; social media studies; young adults.

Figure 1.

Incentive conditions at each survey wave and survey retention rates for the 30- and 60-day follow-ups.