Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Nov 6;1(3):e12042.
doi: 10.1002/jcv2.12042. eCollection 2021 Oct.

Ensemble classification of autism spectrum disorder using structural magnetic resonance imaging features

Yanli Zhang-James  1 , Jan K Buitelaar  2   3   4 ; ENIGMA‐ASD Working Group; Daan van Rooij  5 , Stephen V Faraone  1   6
Affiliations

Ensemble classification of autism spectrum disorder using structural magnetic resonance imaging features

Yanli Zhang-James et al. JCPP Adv. .

Abstract

Background: Autism spectrum disorder (ASD) is characterized by a spectrum of social and communication impairments and rigid and stereotyped behaviors that have a neurodevelopmental origin. Although many imaging studies have reported structural and functional alterations in multiple brain regions, clinically useful diagnostic imaging biomarkers for ASD remain unavailable.

Methods: In this study, we applied machine learning (ML) models to regional volumetric and cortical thickness data from the largest structural magnetic resonance imaging (sMRI) dataset available from the Enhancing Neuro Imaging Genetics Through Meta-Analysis (ENIGMA) consortium (1833 subjects with ASD and 1838 without ASD; age range: 1.5-64; average age: 15.6; male/female ratio: 4.2:1).

Results: The highest classification accuracy on a hold-out test set was achieved using a stacked Extra Tree Classifier. The area under the receiver operating characteristic (ROC) curve (AUC) was 0.62 (95% confidence interval [CI]: 0.57, 0.68) and the area under the precision-recall curve was 0.58. Learning curve analysis showed the good fit of the model and suggests that more training examples will not likely benefit model performance.

Conclusions: Our results suggest that sMRI volumetric and cortical thickness data alone may not provide clinically sufficient useful diagnostic biomarkers for ASD. Developing clinically useful imaging classifiers for ASD will benefit from combining other data modalities or feature types, such as functional MRI data and raw images that can leverage other machine learning (ML) techniques such as convolutional neural networks.

Keywords: MRI; autism spectrum disorder; biomarkers; classification; machine learning.

PubMed Disclaimer

Conflict of interest statement

S.V.F. received income, potential income, travel expenses continuing education support, and/or research support from Takeda, OnDosis, Tris, Otsuka, Arbor, Ironshore, Rhodes, Akili Interactive Labs, Enzymotec, Sunovion, Supernus, and Genomind. With his institution, he has US patent US20130217707 A1 for the use of sodium‐hydrogen exchange inhibitors in the treatment of ADHD. He also receives royalties from books published by Guilford Press: Straight Talk about Your Child's Mental Health, Oxford University Press: Schizophrenia: The Facts and Elsevier: ADHD: Non‐Pharmacologic Interventions. He is Program Director of www.ADHDinAdults.com. He is a member of the Editorial Advisory Board for JCPP Advances. Y.Z‐J. is also a member of the Editorial Advisory Board for JCPP Advances. J.K.B. has been in the past 3 years a consultant to/member of advisory board of/and/or speaker for Takeda/Shire, Roche, Medice, Angelini, Janssen, and Servier. He is not an employee of any of these companies, and not a stock shareholder of any of these companies. He has no other financial or material support, including expert testimony, patents, and royalties. The remaining authors have declared that they have no competing or potential conflicts of interest to declare. [Corrections made on 22 June 2022, after first online publication: This Conflict of Interests statement has been updated in this version.]Dr. Arango has been a consultant to or has received honoraria or grants from Acadia, Angelini, Gedeon Richter, Janssen Cilag, Lundbeck, Minerva, Otsuka, Roche, Sage, Servier, Shire, Schering Plough, Sumitomo Dainippon Pharma, Sunovion and Takeda. Dr. Anagnostou has served as a consultant or advisory board member for Roche and Takeda. Dr. Freitag has served as a consultant for Desitin regarding issues on ASD. Dr. Rubia has received funding from Takeda pharmaceuticals for another project. Dr. Gallagher received funding from the Meath Foundation and the National Children's Research Centre in Ireland. Dr. Parellada has served as a consultant, advisory board member or received honoraria from Sevier and Exeltis. She has received travel support from Janssen Cilag and Lundbeck. Dr. Murphy has served on advisory boards for Roche and Servier. Dr. Franke has received educational speaking fees from Medice. The other authors report no financial relationships with commercial interests.

Figures

FIGURE 1
FIGURE 1
The ensemble machine learning pipeline includes eight base models that were trained as independent classifiers (light blue), followed by various ensemble methods in the second tier (dark blue) to improve the individual models, and the cross‐classifier ensemble in the third tier (navy blue) to combine multiple different classifiers
FIGURE 2
FIGURE 2
Final training, validation and test AUCs were plotted for each classifier using different features. (A) magnetic resonance imaging features. (B) Principal component analysis features. (C) Autoencoded features
See this image and copyright information in PMC

References

    1. Beede, E. , Baylor, E. , Hersch, F. , Iurchenko, A. , Wilcox, L. , Ruamviboonsuk, P. , & Vardoulakis, L. M. (2020). A human‐centered evaluation of a deep learning system deployed in clinics for the detection of diabetic retinopathy. In Paper presented at the Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA. 10.1145/3313831.3376718 - DOI
    1. Bergstra, J. , Yamins, D. , & Cox, D. D. (2013). Making a science of model search: Hyperparameter optimizationin hundredsof dimensions for vision architectures. In Paper presented at the The 30thInternational Conference on Machine Learning (ICML 2013), Atlanta, Gerorgia.
    1. Boedhoe, P. S. W. , van Rooij, D. , Hoogman, M. , Twisk, J. W. R. , Schmaal, L. , Abe, Y. , Alonso, P. , Ameis, S. H. , Anikin, A. , Anticevic, A. , Arango, C. , (2020). Subcortical brain volume, regional cortical thickness, and cortical surface area across disorders: Findings from the ENIGMA ADHD, ASD, and OCD working groups. American Journal of Psychiatry, 177(9), 834–843. 10.1176/appi.ajp.2020.19030331 - DOI - PMC - PubMed
    1. Brain, D. , & Webb, G. I. (1999). On the effect of dataset size on bias and variance in classification learning. In Paper presented at the Proceedings of the Fourth Australian Knowledge Acquisition Workshop (AKAW '99), Sydney.
    1. Davis, J. , & Goadrich, M. (2006). The relationship between Precision‐Recall and ROC curves. In Paper presented at the Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, Pennsylvania, USA. 10.1145/1143844.1143874 - DOI