Developing a Theory of Change for a Digital Youth Mental Health Service (Moderated Online Social Therapy): Mixed Methods Knowledge Synthesis Study

Abstract

Background: Common challenges in the youth mental health system include low access, poor uptake, poor adherence, and limited overall effectiveness. Digital technologies offer promise, yet challenges in real-world integration and uptake persist. Moderated Online Social Therapy (MOST) aims to overcome these problems by integrating a comprehensive digital platform into existing youth mental health services. Theory of change (ToC) frameworks can help articulate how and why complex interventions work and what conditions are required for success.

Objective: The objective of this study is to create a ToC for MOST to explain how it works, why it works, who benefits and how, and what conditions are required for its success.

Methods: We used a multimethod approach to construct a ToC for MOST. The synthesis aimed to assess the real-world impact of MOST, a digital platform designed to enhance face-to-face youth mental health services, and to guide its iterative refinement. Data were gathered from 2 completed and 4 ongoing randomized controlled trials, 11 pilot studies, and over 1000 co-design sessions using MOST. Additionally, published qualitative findings from diverse clinical contexts and a review of related digital mental health literature were included. The study culminated in an updated ToC framework informed by expert feedback. The final ToC was produced in both narrative and table form and captured components common in program logic and ToC frameworks.

Results: The MOST ToC captured several assumptions about digital mental health adoption, including factors such as the readiness of young people and service providers to embrace digital platforms. External considerations included high service demand and a potential lack of infrastructure to support integration. Young people and service providers face several challenges and pain points MOST seeks to address, such as limited accessibility, high demand, poor engagement, and a lack of personalized support. Self-determination theory, transdiagnostic psychological treatment approaches, and evidence-based implementation theories and their associated mechanisms are drawn upon to frame the intervention components that make up the platform. Platform usage data are captured and linked to short-, medium-, and long-term intended outcomes, such as reductions in mental health symptoms, improvements in functioning and quality of life, reductions in hospital visits, and reduced overall mental health care costs.

Conclusions: The MOST ToC serves as a strategic framework for refining MOST over time. The creation of the ToC helped guide the development of therapeutic content personalization, user engagement enhancement, and clinician adoption through specialized implementation frameworks. While powerful, the ToC approach has its limitations, such as a lack of standardized methodology and the amount of resourcing required for its development. Nonetheless, it provides an invaluable roadmap for iterative development, evaluation, and scaling of MOST and offers a replicable model for other digital health interventions aiming for targeted, evidence-based impact.

Keywords: adolescence; adolescent; blended; blended care; co-design; development; digital health; digital intervention; digital mental health; framework; hybrid; mental health; model; platform; self-determination theory; service; services; theory of change; therapy; youth; youth mental health.

Figure 1

Moderated Online Social Therapy (MOST) use cases and intended purpose at different phases of treatment: from before treatment (universal), to waiting for care, to engaging in face-to-face care in services, to being recently discharged from care.