A Machine Learning Approach to Understanding Patterns of Engagement With Internet-Delivered Mental Health Interventions

Abstract

Importance: The mechanisms by which engagement with internet-delivered psychological interventions are associated with depression and anxiety symptoms are unclear.

Objective: To identify behavior types based on how people engage with an internet-based cognitive behavioral therapy (iCBT) intervention for symptoms of depression and anxiety.

Design, setting, and participants: Deidentified data on 54 604 adult patients assigned to the Space From Depression and Anxiety treatment program from January 31, 2015, to March 31, 2019, were obtained for probabilistic latent variable modeling using machine learning techniques to infer distinct patient subtypes, based on longitudinal heterogeneity of engagement patterns with iCBT.

Interventions: A clinician-supported iCBT-based program that follows clinical guidelines for treating depression and anxiety, delivered on a web 2.0 platform.

Main outcomes and measures: Log data from user interactions with the iCBT program to inform engagement patterns over time. Clinical outcomes included symptoms of depression (Patient Health Questionnaire-9 [PHQ-9]) and anxiety (Generalized Anxiety Disorder-7 [GAD-7]); PHQ-9 cut point greater than or equal to 10 and GAD-7 scores greater than or equal to 8 were used to define depression and anxiety.

Results: Patients spent a mean (SD) of 111.33 (118.92) minutes on the platform and completed 230.60 (241.21) tools. At baseline, mean PHQ-9 score was 12.96 (5.81) and GAD-7 score was 11.85 (5.14). Five subtypes of engagement were identified based on patient interaction with different program sections over 14 weeks: class 1 (low engagers, 19 930 [36.5%]), class 2 (late engagers, 11 674 [21.4%]), class 3 (high engagers with rapid disengagement, 13 936 [25.5%]), class 4 (high engagers with moderate decrease, 3258 [6.0%]), and class 5 (highest engagers, 5799 [10.6%]). Estimated mean decrease (SE) in PHQ-9 score was 6.65 (0.14) for class 3, 5.88 (0.14) for class 5, and 5.39 (0.14) for class 4; class 2 had the lowest rate of decrease at -4.41 (0.13). Compared with PHQ-9 score decrease in class 1, the Cohen d effect size (SE) was -0.46 (0.014) for class 2, -0.46 (0.014) for class 3, -0.61 (0.021) for class 4, and -0.73 (0.018) for class 5. Similar patterns were found across groups for GAD-7.

Conclusions and relevance: The findings of this study may facilitate tailoring interventions according to specific subtypes of engagement for individuals with depression and anxiety. Informing clinical decision needs of supporters may be a route to successful adoption of machine learning insights, thus improving clinical outcomes overall.

Figure 1.. The Hidden Markov Model

Y represents a binary response (0 = no, 1 = yes) to the question, “Has patient i engaged with the platform at time t?” where t takes values from 1 to 14. K is a multinomial latent variable per patient and is set in ranges of 2 to 10, π is the starting probability, a_br is the class (K)–dependent transition probabilities of transition from state b to state r, q_s and q’_s are the emission probabilities of observing engagement with a section () conditioned on the hidden state, q_s indicates that the hidden state takes a value of 1, and q’_s indicates that the hidden state takes a value of 0. When sections data are used, emission probabilities are considered distinct for each section.

Figure 2.. Five Distinct Patterns of Engagement With Internet-Based Cognitive Behavioral Therapy Identified

Panel A shows the proportion of users who are engaged for each class at each time point over a 14-week time period. Panels B-F show the proportion of users who use each of the program sections.

Figure 3.. Patterns of Interaction With the Program for Engagement Subtypes

Interactions based on core modules completed (A), mean time spent per week on the program (B), and mean number of sessions per week (sum of session duration) (C). Error bars indicate 95% CIs.