Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2016 Feb:59:299-307.
doi: 10.1016/j.jbi.2015.12.015. Epub 2015 Dec 28.

More than just tracking time: Complex measures of user engagement with an internet-based health promotion intervention

Nina B Baltierra  1 , Kathryn E Muessig  2 , Emily C Pike  1 , Sara LeGrand  3 , Sheana S Bull  4 , Lisa B Hightow-Weidman  1
Affiliations

More than just tracking time: Complex measures of user engagement with an internet-based health promotion intervention

Nina B Baltierra et al. J Biomed Inform. 2016 Feb.

Abstract

Background: There has been a rise in internet-based health interventions without a concomitant focus on new methods to measure user engagement and its effect on outcomes. We describe current user tracking methods for internet-based health interventions and offer suggestions for improvement based on the design and pilot testing of healthMpowerment.org (HMP).

Methods: HMP is a multi-component online intervention for young Black men and transgender women who have sex with men (YBMSM/TW) to reduce risky sexual behaviors, promote healthy living and build social support. The intervention is non-directive, incorporates interactive features, and utilizes a point-based reward system. Fifteen YBMSM/TW (age 20-30) participated in a one-month pilot study to test the usability and efficacy of HMP. Engagement with the intervention was tracked using a customized data capture system and validated with Google Analytics. Usage was measured in time spent (total and across sections) and points earned.

Results: Average total time spent on HMP was five hours per person (range 0-13). Total time spent was correlated with total points earned and overall site satisfaction.

Conclusion: Measuring engagement in internet-based interventions is crucial to determining efficacy. Multiple methods of tracking helped derive more comprehensive user profiles. Results highlighted the limitations of measures to capture user activity and the elusiveness of the concept of engagement.

Keywords: HIV; Internet; Intervention; Men who have sex with men; Mobile phone.

PubMed Disclaimer

Figures

Figure 1
Figure 1
HMP adaptation of the Integrated Behavioral Model
Figure 2
Figure 2
Screenshot of Homepage from desktop and mobile
Figure 3
Figure 3
Total Time spent on intervention site by users
Figure 4
Figure 4
Total Reputation Points earned by users
Figure 5
Figure 5
Reputation Points earned by users each week
Figure 6
Figure 6
Correlation between time spent on site and Reputation Points earned
Figure 7
Figure 7
Correlation between time spent on site and overall site satisfaction
See this image and copyright information in PMC

References

    1. Ajzen I. The theory of planned behavior. Organizational Behavior and Human Decision Processes. 1991;50(2):179–211. doi:http://dx.doi.org/10.1016/0749-5978(91)90020-T.
    1. Allison S, Bauermeister JA, Bull S, Lightfoot M, Mustanski B, Shegog R, Levine D. The intersection of youth, technology, and new media with sexual health: moving the research agenda forward. J Adolesc Health. 2012;51(3):207–212. doi:10.1016/j.jadohealth.2012.06.012. - PMC - PubMed
    1. Brouwer W, Kroeze W, Crutzen R, de Nooijer J, de Vries NK, Brug J, Oenema A. Which intervention characteristics are related to more exposure to internet-delivered healthy lifestyle promotion interventions? A systematic review. Journal of medical Internet research. 2011;13(1):e2. doi:v13i1e2 [pii] 10.2196/jmir.1639. - PMC - PubMed
    1. Bull S, Pratte K, Whitesell N, Rietmeijer C, McFarlane M. Effects of an Internet-based intervention for HIV prevention: the Youthnet trials. AIDS Behav. 2009;13(3):474–487. doi:10.1007/s10461-008-9487-9. - PMC - PubMed
    1. Collins R, Martino S, Shaw S. Influence of new media on adolescent sexual health: evidence and opportunities. 2011. Retrieved from.

Publication types

LinkOut - more resources