Connectome-based predictive modeling of attention: Comparing different functional connectivity features and prediction methods across datasets

Abstract

Connectome-based predictive modeling (CPM; Finn et al., 2015; Shen et al., 2017) was recently developed to predict individual differences in traits and behaviors, including fluid intelligence (Finn et al., 2015) and sustained attention (Rosenberg et al., 2016a), from functional brain connectivity (FC) measured with fMRI. Here, using the CPM framework, we compared the predictive power of three different measures of FC (Pearson's correlation, accordance, and discordance) and two different prediction algorithms (linear and partial least square [PLS] regression) for attention function. Accordance and discordance are recently proposed FC measures that respectively track in-phase synchronization and out-of-phase anti-correlation (Meskaldji et al., 2015). We defined connectome-based models using task-based or resting-state FC data, and tested the effects of (1) functional connectivity measure and (2) feature-selection/prediction algorithm on individualized attention predictions. Models were internally validated in a training dataset using leave-one-subject-out cross-validation, and externally validated with three independent datasets. The training dataset included fMRI data collected while participants performed a sustained attention task and rested (N = 25; Rosenberg et al., 2016a). The validation datasets included: 1) data collected during performance of a stop-signal task and at rest (N = 83, including 19 participants who were administered methylphenidate prior to scanning; Farr et al., 2014a; Rosenberg et al., 2016b), 2) data collected during Attention Network Task performance and rest (N = 41, Rosenberg et al., in press), and 3) resting-state data and ADHD symptom severity from the ADHD-200 Consortium (N = 113; Rosenberg et al., 2016a). Models defined using all combinations of functional connectivity measure (Pearson's correlation, accordance, and discordance) and prediction algorithm (linear and PLS regression) predicted attentional abilities, with correlations between predicted and observed measures of attention as high as 0.9 for internal validation, and 0.6 for external validation (all p's < 0.05). Models trained on task data outperformed models trained on rest data. Pearson's correlation and accordance features generally showed a small numerical advantage over discordance features, while PLS regression models were usually better than linear regression models. Overall, in addition to correlation features combined with linear models (Rosenberg et al., 2016a), it is useful to consider accordance features and PLS regression for CPM.

Keywords: Attention; Functional connectivity; Partial least square regression; Predictive model.

Figure 1. Performance of 12 Connectome-based Predictive Models (CPMs)

CPM performance was assessed by correlating predicted and observed individual scores. The twelve models varied according to: definition of functional connectivity (Pearson’s correlation, accordance, or discordance), feature selection and predictive algorithms (linear or PLS regression), and data type on which the model was trained (gradCPT-based fMRI or resting-state fMRI). All predictive models were validated within the gradCPT dataset and applied to three independent datasets (stop-signal task, ANT, and ADHD). Except external dataset 3, which contains only resting-state fMRI, all datasets include both task-based fMRI and resting-state fMRI. Dark red bars and dotted lines represent results from our previously published model (Rosenberg et al., 2016a, b).

Figure 2. Average performance of twelve connectome-based predictive models (CPMs) in internal and external datasets

(A) Mean performance of all models trained on gradCPT task vs. gradCPT rest data. (B) Mean performance of all linear vs. all PLS regression models. (C) Mean performance of all Pearson’s correlations vs. accordance vs. discordance models. (D) For the internal validation columns, bars represent predictive power averaged across connectivity measures. For the external validation columns, bars represent predictive power averaged across connectivity measures and datasets. For example, the “task external” group came from averaging the results from testing on stop-signal task-based fMRI and testing on ANT-based fMRI. The “rest external” group was obtained from averaging the results from testing on resting-state fMRI from the three external datasets.

Figure 3. Similarity among connectivity measures and between sessions

The first three pairs represent similarities between 1) correlation and accordance, 2) correlation and discordance, and 3) accordance and discordance, all calculated from gradCPT-based fMRI. Pairs 4–6 represent the same pairs as 1–3, but calculated from resting-state fMRI. Pairs 7–9 represent similarities between task fMRI and resting fMRI of 7) correlation, 8) accordance, and 9) discordance.

Figure 3. Edges with significant weights shared across PLS CPMs using different functional connectivity features

This figure shows edges with a significant beta coefficient in the three PLS prediction models using the three different connectivity measures. (A) Anatomical distribution of significant edges in each model; color saturation represents proportion relative to total number of possible edges between each pair of anatomical regions. (B) Edges that had significant weights in all three models. (C) Significant edges are shown in yellow for each model, and common edges across models are aligned vertically. 106 edges (of a possible 35,778) had significant positive weights, and 95 edges had significant negative weights in at least one connectivity measure (two-tailed p < 0.05 based on 106 permutations).

Figure 5. Edges with significant weights in the model using PLS regression to predict attention from Pearson’s correlation coefficients

Bar plots show the number of edges with at least one node in each lobe. Occipital, temporal, and cerebellar regions had the highest number of edges with significant positive weights. Temporal and cerebellar regions also had the highest number of edges with significant negative weights. Every edge is represented twice in this figure – for example, a prefrontal-temporal connection is counted in both the prefrontal and temporal bars.

Figure 6. Comparison between significant edges from PLS and linear regression

(A) PLS regression beta weights in the saCPM’s high-attention network (HAN), low-attention network (LAN), and in neither. (B) Edges with significant PLS weights that are also included in the HAN or LAN are shown in blue; edges with significant PLS weights that are not included in the HAN or LAN are shown in yellow.