Developing personalized algorithms for sensing mental health symptoms in daily life

Abstract

The integration of artificial intelligence (AI) and pervasive computing offers new opportunities to sense mental health symptoms and deliver just-in-time adaptive interventions via mobile devices. This pilot study tested personalized versus generalized machine learning models for detecting individual and family mental health symptoms as a foundational step toward JITAI development, using data collected through the Colliga app on smart devices. Over a 60-day period, data from 35 families resulted in approximately 14 million data points across 52 data streams. Findings showed that personalized models consistently outperformed generalized models. Model performance varied significantly based on individual factors and symptom profiles, underscoring the need for tailored approaches. These preliminary findings suggest that successful implementation of passive sensing technologies for mental health will require accounting for users' unique characteristics. Further research with larger samples is needed to refine the models, address data heterogeneity, and develop scalable systems for personalized mental health interventions.

Fig. 1. Overview of study procedures.

Participants completed an initial phone screening to determine eligibility. Eligible families proceeded to Visit 1 for consent, a second screening to assess for clinical risk factors, and baseline questionnaires (not relevant to the current study). Devices were shipped after qualifying at Visit 1. Participants who completed the initial screen but were ineligible received a screening compensation. After device setup at Visit 2, participants engaged in ~60 days of home-based data collection, completing two daily surveys and audio recordings per day in addition to providing passive phone and wearable sensor data. At Visit 3, participants completed final questionnaires (not relevant to the current study), received their second payment, and shipped back devices.

Fig. 2. Screenshots of the Colliga App used to collect data.

Screenshots of an earlier version of the Colliga App used for data collection in this study. A Includes the researcher-facing web app for study configuration. B Includes the participant-facing mobile app for data collection.

Fig. 3. Number of participants with each data stream and adherence.

A Provides the number of participants with each of the data streams used in model development. B Provides the percentage of days for which there were at least 4 h of data for model development for those participants providing the data stream.

Fig. 4. Time series plots of for one example participant for their endorsement of four target states across the 60-day period.

Happy (A), sad (B), anxious (C), and angry (D) mood for one example participant over the 60-day data collection period. Small pink circle = first survey. Medium yellow circle = second survey. Large blue circle = third survey. Extra-large purple circle = fourth survey.

Fig. 5. Heat map of sample feature associations with the 11 target states for four example participants.

Heat maps for 11 target states and the 20 features with the highest average adherence for the sample over the 60 days for four example participants. A Example Participant 1; B Example Participant 2; C Example Participant 3; D Example Participant 4. Blue = positive correlation. Red = negative correlation. Because data streams varied, each participant has a subset out of the 20 possible. *p < 0.05; **p < 0.01; ***p < 0.001.

Fig. 6. Model performance symptom endorsement, survey adherence, and symptom variability.

Sensitivity, specificity, and F1 scores by symptom endorsement (A), survey adherence (B), and symptom variability (C). Average metric and standard error bars are highlighted in black. Blue = happiness. Yellow = sadness. Bright green = anxiety. Anger = red. Stress = purple. Brown = quality time. Pink = closeness. Gray = positive interactions. Light green = negative interactions. Light blue = conflict. Extra light blue = aggression.