Behavioral Studies Using Large-Scale Brain Networks - Methods and Validations

Abstract

Mapping human behaviors to brain activity has become a key focus in modern cognitive neuroscience. As methods such as functional MRI (fMRI) advance cognitive scientists show an increasing interest in investigating neural activity in terms of functional connectivity and brain networks, rather than activation in a single brain region. Due to the noisy nature of neural activity, determining how behaviors are associated with specific neural signals is not well-established. Previous research has suggested graph theory techniques as a solution. Graph theory provides an opportunity to interpret human behaviors in terms of the topological organization of brain network architecture. Graph theory-based approaches, however, only scratch the surface of what neural connections relate to human behavior. Recently, the development of data-driven methods, e.g., machine learning and deep learning approaches, provide a new perspective to study the relationship between brain networks and human behaviors across the whole brain, expanding upon past literatures. In this review, we sought to revisit these data-driven approaches to facilitate our understanding of neural mechanisms and build models of human behaviors. We start with the popular graph theory approach and then discuss other data-driven approaches such as connectome-based predictive modeling, multivariate pattern analysis, network dynamic modeling, and deep learning techniques that quantify meaningful networks and connectivity related to cognition and behaviors. Importantly, for each topic, we discuss the pros and cons of the methods in addition to providing examples using our own data for each technique to describe how these methods can be applied to real-world neuroimaging data.

Keywords: data driven; graph theory; machine learning; neural network; neuroscience.

FIGURE 1

Typical architectural features of functional brain networks. (A) The simplest model is entirely random structure. (B) Networks with modular structure, divided into communities with dense connectivity. (C) Small-world networks, which balance efficient communication and high clustering. (D) Networks with hub structure, characterized by a heavy-tailed degree distribution.

FIGURE 2

The CPM approach identifies functional connectivity networks that are related to behavior and measures strength in these networks in previously unseen individuals to make predictions about their behavior. First, every participant’s whole-brain connectivity pattern is calculated by correlating the fMRI activity time courses of every pair of regions, or nodes, in a brain atlas. Next, behaviorally relevant connections are identified by correlating every connection in the brain with behavior across subjects. Connections that are most strongly related to behavior in the positive and negative directions are retained for model building. A linear model relates each individual’s positive network strength (i.e., the sum of the connections in their positive network) and negative network strength (i.e., the sum of the connections in their negative network) to their behavioral score. The model is then applied to a novel individual’s connectivity data to generate a behavioral prediction. Predictive power is assessed by correlating predicted and observed behavioral scores across the group.

FIGURE 3

Representational similarity analysis. Illustration of the steps of RSA for a simple design with six visual stimuli. (A) Stimuli (or, more generally, experimental conditions) are assumed to elicit brain representations of individual pieces of content (e.g., visual objects). Here, the representation of each item is visualized as a set of voxels (an fMRI region of interest) that are active to different degrees (black-to-red color scale). We compute the dissimilarity for each pair of stimuli, for example using 1–correlation across voxels. (B) The representational dissimilarity matrix (RDM) assembles the dissimilarities for all pairs of stimuli (blue-to-red color scale for small-to-large dissimilarities). The matrix can be used like a table to look up the dissimilarity between any two stimuli. The RDM is typically symmetric about a diagonal of zeros (white entries along the diagonal). RDMs can similarly be computed from stimulus descriptions (bottom left), from internal representations in computational models (bottom right), and from behavior (top right). By correlating RDMs (black double arrows), we can then assess to what extent the brain representation reflects stimulus properties, can be accounted for by different computational models, and is reflected in behavior. Adapted with permission from Kriegeskorte and Kievit (2013).

FIGURE 4

Hidden Markov Modeling (HMM) network analysis (B) as opposed to sliding-window network analysis (A). Whereas the sliding window has a fixed width and ignores the data beyond its boundaries, the HMM automatically finds, across the entire data set, all the network occurrences that correspond to a given state, enhancing the robustness of the estimation (because it has more data than a window) and adapting to inherent network time in a data driven manner. In this example, the states themselves reflect unique spatial patterns of oscillatory envelopes and envelope couplings, that consistently repeat and different points in time. The non-marked segments of the data correspond to other states.

FIGURE 5

Architecture of a CNN on graphs and the four ingredients of a (graph) convolutional layer.

FIGURE 6

Schematic representation of the BrainNetCNN architecture. Each block represents the input and/or output of the numbered filter layers. The 3rd dimension of each block (i.e., along vector m) represents the number of feature maps, M, at that stage. The brain network adjacency matrix (leftmost block) is first convolved with one or more (two in this case) E2E filters which weight edges of adjacent brain regions. The response is convolved with an E2N filter which assigns each brain region a weighted sum of its edges. The N2G assigns a single response based on all the weighted nodes. Finally, fully connected (FC) layers reduce the number of features down to two output score predictions.

FIGURE 7

Estimation of single subject connectivity matrix and labeled graph representation. Pearson’s correlation is used to obtain a functional connectivity matrix from the raw fMRI time-series. After specifying the graph structure for all subjects, based on spatial or functional information, each row/column of the connectivity matrix serves as a signal for the corresponding node (node features). The common connectivity matrix used for all subjects can be established using the anatomical information, e.g., the spatial distance between brain regions, or the physiological information, e.g., the mean functional connectivity matrix among the training samples.