Predicting adherence to fully-automated, chatbot-delivered digital cognitive behavioral therapy for insomnia (dCBT-I) using machine learning: A pilot real-world study

Abstract

Digital cognitive behavioral therapy for insomnia (dCBT-I) is effective in treating insomnia, but adherence remains a major challenge in real-world applications. Machine learning (ML) offers potential in predicting healthcare utilization. This study applied ML techniques to predict adherence to dCBT-I based on participant baseline characteristics. This pilot real-world study included 75 individuals (69% female; 41% aged 35-44 years) with insomnia symptoms (Insomnia Severity Index, ISI ≥ 8) who participated in a 28-day chatbot-delivered dCBT-I program. ML models, including logistic regression with elastic-net penalty, support vector machine, random forest, and gradient boosting, analyzed participant baseline characteristics to predict adherence to dCBT-I in terms of session completion, usage duration, and response volume. These models were fine-tuned using grid search and evaluated with cross-validation. The synthetic minority over-sampling technique was applied to address data imbalances in the training set. Baseline depressive symptoms were the most predictive of non-adherence. Higher depressive symptoms were associated with shorter overall usage duration (β = -3.57, 95% CI: -5.82 to -1.33, p = .002). Longer sleep onset latency and wake time after sleep onset from the previous night increased the number of responses and longer usage duration on the following day (β = 0.01-0.05, p < .05). No significant associations were found between daytime and bedtime usage and sleep parameters for that specific night. ML models predicted overall adherence, with AUCs of 0.65-0.91 (p < .05). ML models also predicted next-day adherence, with AUCs of 0.56-0.74 (p < .05). This real-world study demonstrates the potential of ML to predict user adherence to dCBT-I and provides clinical insights for personalizing sleep-focused treatments. The study also investigated daily usage and adherence patterns in dCBT-I to predict next-day adherence.

Fig 1. SHapley Additive exPlanations (SHAP) summary for gradient boosting models predicting overall and daily adherence.

ISI: Insomnia Severity Index; rMEQ: Reduced Horne and Östberg Morningness and Eveningness Questionnaire; PHQ-9: Patient Health Questionnaire-9; DBAS: Dysfunctional Beliefs and Attitudes about Sleep; SHI: Sleep Hygiene Index scores; SOL: Sleep onset latency; WASO: Wake time after sleep onset; TIB: Total time spent in bed; TST: Total sleep time; SE: Sleep efficiency.