Combining Artificial Intelligence and Human Support in Mental Health: Digital Intervention With Comparable Effectiveness to Human-Delivered Care

Abstract

Background: Escalating mental health demand exceeds existing clinical capacity, necessitating scalable digital solutions. However, engagement remains challenging. Conversational agents can enhance engagement by making digital programs more interactive and personalized, but they have not been widely adopted. This study evaluated a digital program for anxiety in comparison to external comparators. The program used an artificial intelligence (AI)-driven conversational agent to deliver clinician-written content via machine learning, with clinician oversight and user support.

Objective: This study aims to evaluate the engagement, effectiveness, and safety of this structured, evidence-based digital program with human support for mild, moderate, and severe generalized anxiety. Statistical analyses sought to determine whether the program reduced anxiety more than a propensity-matched waiting control and was statistically noninferior to real-world, propensity-matched face-to-face and typed cognitive behavioral therapy (CBT).

Methods: Prospective participants (N=299) were recruited from the National Health Service (NHS) or social media in the United Kingdom and given access to the digital program for up to 9 weeks (study conducted from October 2023 to May 2024). End points were collected before, during, and after the digital program, as well as at a 1-month follow-up. External comparator groups were created through propensity matching of the digital program sample with NHS Talking Therapies (NHS TT) data from ieso Digital Health (typed CBT) and Dorset HealthCare (DHC) University NHS Foundation Trust (face-to-face CBT). Superiority and noninferiority analyses were conducted to compare anxiety symptom reduction (change on the 7-item Generalized Anxiety Disorder Scale [GAD-7]) between the digital program group and the external comparator groups. The program included human support, and clinician time spent per participant was calculated.

Results: Participants used the program for a median of 6 hours over 53 days, with 232 of the 299 (77.6%) engaged (ie, completing a median of 2 hours over 14 days). There was a large, clinically meaningful reduction in anxiety symptoms for the digital program group (per-protocol [PP; n=169]: mean GAD-7 change -7.4, d=1.6; intention-to-treat [ITT; n= 99]: mean GAD-7 change -5.4, d=1.1). The PP effect was statistically superior to the waiting control (d=1.3) and noninferior to the face-to-face CBT group (P<.001) and the typed CBT group (P<.001). Similarly, for the ITT sample, the digital program showed superiority to waiting control (d=0.8) and noninferiority to face-to-face CBT (P=.002), with noninferiority to typed CBT approaching significance (P=.06). Effects were sustained at the 1-month follow-up. Clinicians overseeing the digital program spent a mean of 1.6 hours (range 31-200 minutes) of clinician time in sessions per participant.

Conclusions: By combining AI and human support, the digital program achieved clinical outcomes comparable to human-delivered care, while significantly reducing the required clinician time by up to 8 times compared with global care estimates. These findings highlight the potential of technology to scale evidence-based mental health care, address unmet needs, and ultimately impact quality of life and reduce the economic burden globally.

Trial registration: ISRCTN Registry ISRCTN52546704; http://www.isrctn.com/ISRCTN52546704.

Keywords: AI; anxiety; digital intervention; external control; human-in-loop; mental health; smartphone; synthetic control.

Figure 1

Enrollment pathways differed for external recruits (left) and patients referred to ieso for typed therapy (right), either through NHS providers or self-referral. External recruits enrolled specifically for the study via a web page, accessed through social media or email advertisements. All potential participants, regardless of recruitment source, were screened using a Self-Assessment Questionnaire (SAQ). Only patients deemed potentially eligible were invited to participate. Participants were withdrawn for various reasons: actively (upon request), passively (dropout or disengagement), clinician-led (based on clinical judgment), or other (eg, technical issues). GAD-7: 7-item Generalized Anxiety Disorder Scale; TAU: treatment as usual.

Figure 2

Screenshots of the ieso Digital Program user interface.

Figure 3

Schematic of the ieso Digital Program with human clinical and user support service and study procedures. All participants received a clinical assessment before enrollment and were offered a discharge appointment with a clinician following the program. Clinicians were available via asynchronous messaging or for a review appointment whenever needed. All participants received email or SMS text message reminders and fortnightly check-in calls throughout the program to maximize engagement delivered via the research team. The ieso Digital Program included 6 modules with a total of 16 sessions. Each module started with a symptom check-in consisting of the 7-item Generalized Anxiety Disorder Scale and 9-item Patient Health Questionnaire.

Figure 4

Adherence with program progression overlaid with adherence across therapy sessions for the control groups. For each group, adherence was defined based on the proportion of participants who completed each 7-item Generalized Anxiety Disorder Scale (GAD-7) assessment (symptom check) throughout their journey. The baseline was 100%, that is, all participants/patients attended a clinical assessment and had a baseline GAD-7 score. For the ieso Digital Program group, each symptom check-in was at the beginning of each module within the program software (a total of 6 instances in the program). To complete each symptom check-in within the program, participants had to finish the previous module. For the therapy control groups, patients completed each GAD-7 assessment as part of each attended treatment session (either face-to-face or typed) up to 6 treatment sessions. Within NHS Talking Therapies, every attended treatment session includes a GAD-7 assessment. Adherence rates across sessions were not significantly different between groups (see Table S3 in Multimedia Appendix 1). CBT: cognitive behavioral therapy.

Figure 5

Change in anxiety symptoms from baseline to final score for the intervention sample and propensity-matched control groups. (A) Mean change (final score – baseline) in 7-item Generalized Anxiety Disorder Scale (GAD-7) scores for the per-protocol (PP) sample (n=169), propensity-matched waiting control group, face-to-face cognitive behavioral therapy (CBT) group, and typed CBT group. (B) Mean change in GAD-7 scores for the intention-to-treat (ITT) sample (n=299) and all control groups. (C) Mean GAD-7 scores at baseline and final score with 95% CIs for the PP sample (n=169) and all control groups. (D) Mean GAD-7 scores at baseline and final score with 95% CIs for the ITT sample (n=299) and all control groups. **P<.005; ***P<.001.

Figure 6

Mean reduction in anxiety symptoms across the digital program. Mean 7-item Generalized Anxiety Disorder Scale (GAD-7) score for each time point for all participants that completed the questionnaires at each time point. Trajectories split by GAD-7 baseline severity: mild, moderate, and severe (see Table S7 in Multimedia Appendix 1).