Providing context: Extracting non-linear and dynamic temporal motifs from brain activity

Abstract

Approaches studying the dynamics of resting-state functional magnetic resonance imaging (rs-fMRI) activity often focus on time-resolved functional connectivity (tr-FC). While many tr-FC approaches have been proposed, most are linear approaches, e.g. computing the linear correlation at a timestep or within a window. In this work, we propose to use a generative non-linear deep learning model, a disentangled variational autoencoder (DSVAE), that factorizes out window-specific (context) information from timestep-specific (local) information. This has the advantage of allowing our model to capture differences at multiple temporal scales. We find that by separating out temporal scales our model's window-specific embeddings, or as we refer to them, context embeddings, more accurately separate windows from schizophrenia patients and control subjects than baseline models and the standard tr-FC approach in a low-dimensional space. Moreover, we find that for individuals with schizophrenia, our model's context embedding space is significantly correlated with both age and symptom severity. Interestingly, patients appear to spend more time in three clusters, one closer to controls which shows increased visual-sensorimotor, cerebellar-subcortical, and reduced cerebellar-visual functional network connectivity (FNC), an intermediate station showing increased subcortical-sensorimotor FNC, and one that shows decreased visual-sensorimotor, decreased subcortical-sensorimotor, and increased visual-subcortical domains. We verify that our model captures features that are complementary to - but not the same as - standard tr-FC features. Our model can thus help broaden the neuroimaging toolset in analyzing fMRI dynamics and shows potential as an approach for finding psychiatric links that are more sensitive to individual and group characteristics.

Fig 1. An abstract depiction of our model, for the independent version of the model, the context representation is not concatenated inside the local encoder.

Fig 2. The window classification accuracy for two of our proposed models (DSVAE, IDSVAE), windowed FNC (wFNC), and two baseline methods: context-only (CVAE), and local-only (LVAE).

Our proposed methods outperform all other methods, experiments are performed across 4 seeds. Significance levels of the independent t-test: *: p<0.05, **: p<0.01.

Fig 3. Each subfigure shows a different local size (LS) and context size (CS) configuration, where the reliability of the model across different initializations is measured in R-squared.

In most cases where both the local size and context size is small, the DSVAE model is significantly more reliable than the IDSVAE method. However, for a local size of 8, the IDSVAE method is significantly more reliable when the context sizes are 2 or 8. Moreover, reliability decreases with a larger dimensionality of the context space, likely increasing the dimensions essentially increases the size and thus the number of equivalent solutions of the space. Significance levels of the independent t-test: *: p<0.05, **: p<0.01, ***: p<0.001, and ****: p<0.0001.

Fig 4. These subfigures show a visualization of the context embeddings for the DSVAE model with LS=2=4,CS=2. The first subfigure from the left shows three patient clusters, the second subfigure both patients and controls, the third colors context embeddings based on the subject’s age, and the last subfigure colors the context embeddings based on the subject’s cognitive score, CMINDs.

Fig 5. The three schizophrenia patient clusters visualized using functional connectivity matrices. To create the visualizations, we take all the windows belonging to a cluster and average their wFNC matrix.