Resting-state test-retest reliability of a priori defined canonical networks over different preprocessing steps

Abstract

Resting-state functional connectivity analysis has become a widely used method for the investigation of human brain connectivity and pathology. The measurement of neuronal activity by functional MRI, however, is impeded by various nuisance signals that reduce the stability of functional connectivity. Several methods exist to address this predicament, but little consensus has yet been reached on the most appropriate approach. Given the crucial importance of reliability for the development of clinical applications, we here investigated the effect of various confound removal approaches on the test-retest reliability of functional-connectivity estimates in two previously defined functional brain networks. Our results showed that gray matter masking improved the reliability of connectivity estimates, whereas denoising based on principal components analysis reduced it. We additionally observed that refraining from using any correction for global signals provided the best test-retest reliability, but failed to reproduce anti-correlations between what have been previously described as antagonistic networks. This suggests that improved reliability can come at the expense of potentially poorer biological validity. Consistent with this, we observed that reliability was proportional to the retained variance, which presumably included structured noise, such as reliable nuisance signals (for instance, noise induced by cardiac processes). We conclude that compromises are necessary between maximizing test-retest reliability and removing variance that may be attributable to non-neuronal sources.

Keywords: Confound removal; Reliability; Resting-state functional connectivity; Test–retest; fMRI.

Figure 1

Percentage of studies using a certain confound removal method (i.e., White matter and Cerebral spinal fluid signal regression (WMCSF), Global signal regression (GSR), Principle component analysis based corrections (PCA), Tissue signal regression (TSR), Physiological recordings based corrections (Physiological Correction), Independent component analysis based corrections (ICA) and other correction methods such as ANATICOR or grey matter atrophy regression (Others)). The colors represent the interactions of each method with other methods. The first fraction of section which is consistent over the approaches, represented with the word “Only” (in blue) shows the percentage of studies performing only a certain confound removal without any interactions. Additional colors assigned to the other confound removal appears only when there is an interaction. Of note, the interactions of motion regression with other methods are not explicitly shown in Figure 1. However, almost all the studies involved in this literature survey, have removed the motion effects along with the other confound removal approaches demonstrated in the figure.

Figure 2

Nodes of meta-analytically derived networks used for the reliability assessment. A: the core working memory network (Rottschy et al., 2012); B: the extended socio-affective default mode network (Amft et al., 2015)

Figure 3

Pipeline of the entire preprocessing steps until the RS-FC computation: The assessed combinations (inside the red dotted box) indicate the signal processing methods for which the reliability is evaluated in three different domains (I. Extraction of time series, II. PCA-denoising, III. Global signal removal).

Figure 4

Percentage of voxels that overlap between the individual and group masks, relative to the GrpGM for each of the 21 seed regions

Figure 5

Indices of the reliability: The 4 indices of reliability used here are illustrated. A&B shows functional connectivity at two time points (A) at connection level, i.e. for all connections within a given subject (B) at subject level, i.e. for all the subjects within a given connection. (Here between left and right anterior insula (LaIns - RaIns)). C&D represent absolute differences of functional connectivity scores between the two sessions (C) at the connection level, i.e. the mean of the absolute differences over subjects for the 210 connections, and (D) at the subject level, i.e. the mean of the absolute differences over connections for the 42 subjects. E illustrates the variance within the BOLD signal time series of the left anterior Insula for two different combinations of signal processing methods (“GrpGM NoPCA NoGSR” (black), “NoGM PCA TSR” (red)).

Figure 6

Combined rankings of the test-retest reliability at the subject and connection level for Kendall’s correlations and absolute differences. The “Within Networks” ranking refers to intra-network connections of the working memory and the default mode network and the “Between Networks” to inter-network connections. The grey bar represents the summed ranks for the respective categories.

Figure 7

Summary rankings for RoSO and RoCO. Reliability for within network (WMN and eSAD) and between networks is shown separately each combining Kendall’s correlations and absolute difference. The grey bar represents the summed ranks for the respective categories.

Figure 8

Summary rankings of reliability across Kendall’s correlations and absolute differences as well as RoSO and RoCO, separately for within (WMN and eSAD) and between networks. The grey bar represents the summed ranks for the respective categories.

Figure 9

The variance left within the time series (far left column) and the percentage of positive correlations (columns on the far right) for both within and between networks arranged by the overall ranking of the reliability. The plots on the right side exemplify the difference of the distribution of the connectivity scores at different combinations (“GrpGM NoPCA NoGSR” (Top), “GrpGM NoPCA WMCSF” (bottom))