Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Randomized Controlled Trial
. 2010 Nov 18;12(4):e52.
doi: 10.2196/jmir.1430.

Engagement and retention: measuring breadth and depth of participant use of an online intervention

Mick P Couper  1 , Gwen L Alexander, Nanhua Zhang, Roderick J A Little, Noel Maddy, Michael A Nowak, Jennifer B McClure, Josephine J Calvi, Sharon J Rolnick, Melanie A Stopponi, Christine Cole Johnson
Affiliations
Randomized Controlled Trial

Engagement and retention: measuring breadth and depth of participant use of an online intervention

Mick P Couper et al. J Med Internet Res. .

Abstract

Background: The Internet provides us with tools (user metrics or paradata) to evaluate how users interact with online interventions. Analysis of these paradata can lead to design improvements.

Objective: The objective was to explore the qualities of online participant engagement in an online intervention. We analyzed the paradata in a randomized controlled trial of alternative versions of an online intervention designed to promote consumption of fruit and vegetables.

Methods: Volunteers were randomized to 1 of 3 study arms involving several online sessions. We created 2 indirect measures of breadth and depth to measure different dimensions and dynamics of program engagement based on factor analysis of paradata measures of Web pages visited and time spent online with the intervention materials. Multiple regression was used to assess influence of engagement on retention and change in dietary intake.

Results: Baseline surveys were completed by 2513 enrolled participants. Of these, 86.3% (n = 2168) completed the follow-up surveys at 3 months, 79.6% (n = 2027) at 6 months, and 79.4% (n = 1995) at 12 months. The 2 tailored intervention arms exhibited significantly more engagement than the untailored arm (P < .01). Breadth and depth measures of engagement were significantly associated with completion of follow-up surveys (odds ratios [OR] = 4.11 and 2.12, respectively, both P values < .001). The breadth measure of engagement was also significantly positively associated with a key study outcome, the mean increase in fruit and vegetable consumption (P < .001).

Conclusions: By exploring participants' exposures to online interventions, paradata are valuable in explaining the effects of tailoring in increasing participant engagement in the intervention. Controlling for intervention arm, greater engagement is also associated with retention of participants and positive change in a key outcome of the intervention, dietary change. This paper demonstrates the utility of paradata capture and analysis for evaluating online health interventions.

Trial registration: NCT00169312; http://clinicaltrials.gov/ct2/show/NCT00169312 (Archived by WebCite at http://www.webcitation.org/5u8sSr0Ty).

PubMed Disclaimer

Conflict of interest statement

None declared

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Griffiths F, Lindenmeyer A, Powell J, Lowe P, Thorogood M. Why are health care interventions delivered over the internet? A systematic review of the published literature. J Med Internet Res. 2006;8(2):e10. doi: 10.2196/jmir.8.2.e10. http://www.jmir.org/2006/2/e10/v8i2e10 - DOI - PMC - PubMed
    1. Strecher V. Internet methods for delivering behavioral and health-related interventions (eHealth) Annu Rev Clin Psychol. 2007;3:53–76. doi: 10.1146/annurev.clinpsy.3.022806.091428. - DOI - PubMed
    1. Li D, Browne GJ, Wetherbe JC. Why do internet users stick with a specific web site? A relationship perspective. International Journal of Electronic Commerce. 2006;10(4):105–141.
    1. Lin JCC. Online stickiness: its antecedents and effect on purchasing intention. Behaviour & Information Technology. 2007;26(6):507–516.
    1. Glasgow RE, Nelson CC, Kearney KA, Reid R, Ritzwoller DP, Strecher VJ, Couper MP, Green B, Wildenhaus K. Reach, engagement, and retention in an Internet-based weight loss program in a multi-site randomized controlled trial. J Med Internet Res. 2007;9(2):e11. doi: 10.2196/jmir.9.2.e11. http://www.jmir.org/2007/2/e11/v9i2e11 - DOI - PMC - PubMed

Publication types

Associated data