Machine Learning-Based Prediction of Attention-Deficit/Hyperactivity Disorder and Sleep Problems With Wearable Data in Children

Abstract

Importance: Early detection of attention-deficit/hyperactivity disorder (ADHD) and sleep problems is paramount for children's mental health. Interview-based diagnostic approaches have drawbacks, necessitating the development of an evaluation method that uses digital phenotypes in daily life.

Objective: To evaluate the predictive performance of machine learning (ML) models by setting the data obtained from personal digital devices comprising training features (ie, wearable data) and diagnostic results of ADHD and sleep problems by the Kiddie Schedule for Affective Disorders and Schizophrenia Present and Lifetime Version for Diagnostic and Statistical Manual of Mental Disorders, 5th edition (K-SADS) as a prediction class from the Adolescent Brain Cognitive Development (ABCD) study.

Design, setting, and participants: In this diagnostic study, wearable data and K-SADS data were collected at 21 sites in the US in the ABCD study (release 3.0, November 2, 2020, analyzed October 11, 2021). Screening data from 6571 patients and 21 days of wearable data from 5725 patients collected at the 2-year follow-up were used, and circadian rhythm-based features were generated for each participant. A total of 12 348 wearable data for ADHD and 39 160 for sleep problems were merged for developing ML models.

Main outcomes and measures: The average performance of the ML models was measured using an area under the receiver operating characteristics curve (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). In addition, the Shapley Additive Explanations value was used to calculate the importance of features.

Results: The final population consisted of 79 children with ADHD problems (mean [SD] age, 144.5 [8.1] months; 55 [69.6%] males) vs 1011 controls and 68 with sleep problems (mean [SD] age, 143.5 [7.5] months; 38 [55.9%] males) vs 3346 controls. The ML models showed reasonable predictive performance for ADHD (AUC, 0.798; sensitivity, 0.756; specificity, 0.716; PPV, 0.159; and NPV, 0.976) and sleep problems (AUC, 0.737; sensitivity, 0.743; specificity, 0.632; PPV, 0.036; and NPV, 0.992).

Conclusions and relevance: In this diagnostic study, an ML method for early detection or screening using digital phenotypes in children's daily lives was developed. The results support facilitating early detection in children; however, additional follow-up studies can improve its performance.

Figure 1.. Study Population and Formation of Training Dataset

Attention-deficit/hyperactivity disorder (A) and sleep problems (B) populations. The final training data inclusion represents the generated wearable data linked (merged) with each corresponding diagnosis in individuals following a unique identifier. ABCD indicates Adolescent Brain Cognitive Development; ADHD, attention-deficit/hyperactivity disorder.

Figure 2.. Area Under the Receiver Operating Characteristic (AUC) Curves of Target Diagnoses Prediction

Predictions for attention-deficit/hyperactivity disorder (A) and sleep problems (B). The dashed line presents the average performance of the machine learning model. AUC (95% CI) findings for the ADHD population were 0.739 (0.736-0.742) for random forest, 0.789 (0.788-0.790) for extreme gradient boosting (XGBoost), 0.791 (0.790-0.792) for light gradient-boosting machine (LightGBM), and 0.798 for best model (LightGBM). AUC (95% CI) findings for the sleep problems population were 0.704 (0.701-0.707) for random forest, 0.717 (0.714-0.719) for XGBoost, 0.726 (0.723-0.730) for LightGBM, and 0.737 for best model (LightGBM). Each 95% CI was calculated by the list of AUC values from machine learning models.

Figure 3.. Summary of Shapley Additive Explanations Importance by the Best Prediction Model

The lists of feature importance for attention-deficit/hyperactivity disorder (A) and sleep problems (B) are arranged in descending order. The higher value (average importance) indicates that the feature affects the model output (probability) to be higher. BMR indicates basal metabolic rate; Harris, Harris-Benedict equation for calculating BMR; HR, heart rate; IS, interdaily stability; M10, most active 10-hour period; MESOR, midline estimating statistics of rhythm; METs, metabolic equivalents; and REM, rapid eye movement.