DeepASD: a deep adversarial-regularized graph learning method for ASD diagnosis with multimodal data

Abstract

Autism Spectrum Disorder (ASD) is a prevalent neurological condition with multiple co-occurring comorbidities that seriously affect mental health. Precisely diagnosis of ASD is crucial to intervention and rehabilitation. A single modality may not fully reflect the complex mechanisms underlying ASD, and combining multiple modalities enables a more comprehensive understanding. Here, we propose, DeepASD, an end-to-end trainable regularized graph learning method for ASD prediction, which incorporates heterogeneous multimodal data and latent inter-patient relationships to better understand the pathogenesis of ASD. DeepASD first learns cross-modal feature representations through a multimodal adversarial-regularized encoder, and then constructs adaptive patient similarity networks by leveraging the representations of each modality. DeepASD exploits inter-patient relationships to boost the ASD diagnosis that is implemented by a classifier compositing of graph neural networks. We apply DeepASD to the benchmarking Autism Brain Imaging Data Exchange (ABIDE) data with four modalities. Experimental results show that the proposed DeepASD outperforms eight state-of-the-art baselines on the benchmarking ABIDE data, showing an improvement of 13.25% in accuracy, 7.69% in AUC-ROC, and 17.10% in specificity. DeepASD holds promise for a more comprehensive insight of the complex mechanisms of ASD, leading to improved diagnosis performance.

Fig. 1. The workflow of DeepASD.

a Multimodal data for ASD diagnosis. The left panel shows that the multimodal datasets consist of fMRI, automated anatomical quality assessment metrics (ANAT), automated functional quality assessment metrics (FUNC), and demographic information (PHENO). According to the information of the multimodal data, DeepASD constructs a multimodal patient similarity network as shown in the middle panel. Using the weighted fusion method (weights are automatically learned), we obtain a global patient similarity network from the multimodal patient similarity network. The right panel presents the global patient similarity matrix (taking six patients as an example). The darker the color, the more similar of the patient embeddings. b The proposed DeepASD framework. DeepASD first adopts an adversarial-regularized encoder module to align the embeddings from different modalities, thereby the learned embeddings will be aligned into the same latent space. We then construct a patient similarity graph for each modality, where each graph node denotes one patient and the edges denote the inter-patient connections. After that, we fuse the multiple constructed modality-specific graphs into a global graph that represents the patient similarity globally. Finally, we employ a graph neural network classifier for ASD diagnosis based on the inter-patient global graph and aligned embeddings. c Model validation. We compare the proposed DeepASD with six state-of-the-art baselines, including three traditional machine learning-based models and three deep learning-based models, on two benchmarking datasets. The proposed DeepASD outperforms all baselines significantly.

Fig. 2. Performance of DeepASD on ABIDE A and ABIDE B.

Panels a and b illustrate the classification performance in a two-dimensional manner on the two datasets. Panel c shows the ROC curves of DeepASD compared with state-of-the-art baselines on the ABIDE A dataset. The mean ROC curve and standard deviation of 10-fold cross-validation results are shown as bold lines and shad regions, respectively. Panel d reports the performance margin (in terms of ACC, AUC, SEN, and SPE), showing DeepASD outperforms other baselines. Each dot denotes a fold among the 10-fold cross-validation.

Fig. 3. Visualization of cosine similarity heatmap across patients.

The left column shows the results of clustering by sex (a), while the middle column is the results of clustering by age in the ABIDE A dataset (b). The first row represents the similarity matrices on raw features. The second row shows the similarity matrices of multimodal fused features after the multimodal adversarial-regularized encoder (details in Fig. 1b; Section “Multimodal adversarial-regularized encoder”. The last row presents the similarity matrices of fused features through GNN. There is a significant age and sex bias in the diagnosis of ASD, and the results show that the contrast between the diagnostics and the presentation of sex and age clusters, indicating that our multimodal adversarial-regularized encoder and multi-graph fusion GNN module are capable of learning more representative features for diagnosis. c–e are visualizations (t-SNE) of the feature representations in ABIDE A. In the three panels, the red color denotes ASD while green colors denotes TC. c Visualization of the raw features for each modality. d Visualization of raw features and learned features. The learned features are taken after the multimodal adversarial-regularized encoder. It’s clearly observed that the learned features are more distinguishable than the raw features. e Visualization of the learned features for each modality. Compared to the raw feature (c), the learned feature space (e) demonstrate that the fMRI has obvious class separability after feature extraction. In addition, the PHENO modality is also separable while the modalities of ANAT and FUNC are less divisible. The observation validates our explanation on the importance of modalities (Section “Multiple modalities enable more accurate diagnosis”; Fig. 4).

Fig. 4. Visualization of modality importance.

a The performance of training DeepASD on the multi-modal data and single-modality (PHENO, ANAT, FUNC, and fMRI, respectively). b Classification performance using different combinations of the four modalities (PHENO, ANAT, FUNC, FMRI) on the ABIDE A. The classifiers that contain the fMRI modality significantly outperform those without the fMRI modality, highlighting the significance of fMRI in the diagnostic process. Additionally, we also observe that combining auxiliary modalities with fMRI is able to further enhance its performance in ASD diagnosis. c Classification performance using different combinations of the four modalities (PHENO, AAL, DOS, CC200) on the ABIDE B. Though the classification results using the three brain atlas fMRI (AAL, DOS, CC200) are promising, the results are further improved by adding the PHENO modality data. d, e Illustration of the weight of each modality (on the ABIDE A and ABIDE B, respectively) in multi-graph fusion. The results show that fMRI is the most significant contributor in both datasets, which aligns with the results shown in b and c.