User journey method: a case study for improving digital intervention use measurement

Abstract

Background: Many digital mental health interventions meet low levels of use. However, current use measurement methods do not necessarily help identify which intervention elements are associated with dropout, despite this information potentially facilitating iterative intervention development. Here, we suggest improving the comprehensiveness of intervention use measurement with the user journey method, which evaluates every intervention element to identify intervention-specific use barriers.

Methods: We applied user journey method in a clinical trial that investigated the efficacy of a novel game-based intervention, Meliora, for adult Major Depressive Disorder. We modelled the intervention for its four technological (Recruitment, Website, Questionnaires, Intervention Software) and two interpersonal elements (Assessment, Support). We then applied the user journey method to measure how many users proceeded from one element to the next combining social media analytics, website use data, signup data, clinical subject coordinator interview data, symptom questionnaire data, and behavioral intervention use data. These measurements were complemented with the qualitative analysis of the study discovery sources and email support contacts.

Results: Recruitment: The intervention recruitment reached at least 145,000 Finns, with social media, word-of-mouth, and news and web sources being the most effective recruitment channels. Website: The study website received 16,243 visitors, which led to 1,007 sign-ups.

Assessment: 895 participants were assessed and 735 were accepted. Intervention Software: 498 participants were assigned to the active intervention or comparator, of whom 457 used them at least once: on average, for 17.3 h (SD = 20.4 h) on 19.7 days (SD = 20.7 d) over a period of 38.9 days (SD = 31.2 d). The 28 intervention levels were associated with an average dropout rate of 2.6%, with two sections exhibiting an increase against this baseline. 150 participants met the minimum adherence goal of 24 h use. Questionnaires: 116 participants completed the post-intervention questionnaire.

Support: 313 signed-up participants contacted the researchers via email.

Conclusion: The user journey method allowed for the comprehensive evaluation of the six intervention elements, and enabled identifying use barriers expediting iterative intervention development and implementation.

Trial registration: ClinicalTrials.gov, NCT05426265. Registered 28 June 2022, https://clinicaltrials.gov/ct2/show/NCT05426265 .

Keywords: Acceptability; Depression; Digital interventions; Engagement; Evaluation methods; Implementation; Mental health; Mixed methods; Use data; User-centered design.

Fig. 1

The modelled user journey of Meliora. The game-based digital intervention consists of six elements and the process that integrates the distinct elements into a service

Fig. 2

The intervention software log-in screen introduces the intervention aesthetics. The translation of the Finnish text is: “Insert here the game credentials you have received via email. Username. Password. Sign in. [✓] Remember me. Did you forget your password? Contact meliora@aalto.fi”

Fig. 3

The user journey. Panel A describes the number of users through the intervention stages, elements, and phases. Panel B describes the proportion of users passing from the previous phase to the next. Support contacts could occur pre-, intra-, and post-intervention

Fig. 4

The study recruitment activities led to variable and short-lived increases in website visitors and sign-ups. Mielenterveystalo.fi is a Finnish digital mental well-being hub, Reddit is a social media platform, Assembly.fi is a Finnish digital gaming community, Aalto.fi is the website of the university the study is conducted in, Yle is the National Broadcasting Company in Finland, and MTV is a commercial Finnish news channel

Fig. 5

The frequency and quantity of intervention use exhibits significant variance. Ten representative participants (#) are selected by their total intervention use hours at decile increments. The number of days (d) during which the participant interacted with the intervention are described per intervention period week. Darker colors indicate more frequent interactions (white = 0 d / wk; light red = 1–2 d / wk; medium red 3–4 d / wk; dark red = 5–7 d / wk)