Can we diagnose mental disorders in children? A large-scale assessment of machine learning on structural neuroimaging of 6916 children in the adolescent brain cognitive development study

Abstract

Background: Prediction of mental disorders based on neuroimaging is an emerging area of research with promising first results in adults. However, research on the unique demographic of children is underrepresented and it is doubtful whether findings obtained on adults can be transferred to children.

Methods: Using data from 6916 children aged 9-10 in the multicenter Adolescent Brain Cognitive Development study, we extracted 136 regional volume and thickness measures from structural magnetic resonance images to rigorously evaluate the capabilities of machine learning to predict 10 different psychiatric disorders: major depressive disorder, bipolar disorder (BD), psychotic symptoms, attention deficit hyperactivity disorder (ADHD), oppositional defiant disorder, conduct disorder, post-traumatic stress disorder, obsessive-compulsive disorder, generalized anxiety disorder, and social anxiety disorder. For each disorder, we performed cross-validation and assessed whether models discovered a true pattern in the data via permutation testing.

Results: Two of 10 disorders can be detected with statistical significance when using advanced models that (i) allow for non-linear relationships between neuroanatomy and disorder, (ii) model interdependencies between disorders, and (iii) avoid confounding due to sociodemographic factors: ADHD (AUROC = 0.567, p = 0.002) and BD (AUROC = 0.551, p = 0.002). In contrast, traditional models perform consistently worse and predict only ADHD with statistical significance (AUROC = 0.529, p = 0.002).

Conclusion: While the modest absolute classification performance does not warrant application in the clinic, our results provide empirical evidence that embracing and explicitly accounting for the complexities of mental disorders via advanced machine learning models can discover patterns that would remain hidden with traditional models.

Keywords: ABCD study; confounding; machine learning; mental disorders; neuroimaging.

FIGURE 1

Flow diagram of participant selection.

FIGURE 2

Model evaluation pipeline. Area under the receiver operating characteristic curve (AUROC) values are obtained by evaluating trained models on the test set, then averaged over all cross‐validation folds. This results in one average AUROC for the original dataset and a distribution of 500 average AUROC representing the distribution under the null hypothesis of “no real pattern has been discovered”.

FIGURE 3

Participants per geographical study site.

FIGURE 4

Comorbidities of studied mental disorders. (top) Studied mental health conditions with overall prevalence; (bottom) 14 most common patterns of disorders and their prevalence. Having no disorder at all was most common (63.7% of participants).

FIGURE 5

Violin plots of cross‐validation results. For each disorder and both classifiers, the distribution of area under the receiver operating characteristic curve (AUROC) under the nullhypothesis of “no real pattern has been discovered” (in gray) is contrasted with the AUROC value (diamond) on the original dataset. Dashed diamonds show AUROC values on unpermuted data with no adjustment by sociodemographic confounders (see Table S5 for a statistical comparison with original AUROC values). Dashed line at $\overline{AUROC}=0.5$ corresponds to a classifier with no discriminative ability. CCE, Gradient boosting model classifier chain ensemble; GBM‐LRC, Logistic regression classifier.