Impact of survey length and compensation on validity, reliability, and sample characteristics for Ultrashort-, Short-, and Long-Research Participant Perception Surveys

Abstract

Introduction: The validated Research Participant Perception Survey (RPPS-Long) elicits valuable data at modest response rates.

Methods: To address this limitation, we developed shorter RPPS-Ultrashort and RPPS-Short versions, fielded them with the RPPS-Long to a random sample of a national research volunteer registry, and assessed response and completion rates, test/retest reliability, and demographics.

Results: 2228 eligible registry members received survey links. Response rates were 64% (RPPS-Ultrashort), 63% (RPPS-Short), and 51% (RPPS-Long), respectively (P<0.001). Completion rates were 63%, 54%, and 37%, respectively (p<0.001). All surveys were reliable with Cronbach's alpha = 0.81, 0.84, and 0.87, respectively. Retest reliability was highest for RPPS-Short (Kappa=0.85). Provision of compensation increased RPPS-Short completion rate from 54% to 71% (p<0.001). Compensated respondents were younger (p<0.001), with greater minority representation (p=0.03).

Conclusions: Shorter surveys were reliable and produced higher response and completion rates then long surveys. Compensation further increased completion rates and shifted sample age and race profiles.

Fig. 1

Participant experience outcomes: frequencies of top box responses to overall rating, would recommend, and 6 key questions, in common across the Research Participant Perception Survey (RPPS)-Ultrashort (RPPS-U), RPPS-Short (RPPS-S), and RPPS-Long (RPPS-L) survey versions are shown. Full question text is provided in the online Supplementary Appendix S1.