Challenges in the reproducibility of clinical studies with resting state fMRI: An example in early Parkinson's disease

Abstract

Resting state fMRI (rfMRI) is gaining in popularity, being easy to acquire and with promising clinical applications. However, rfMRI studies, especially those involving clinical groups, still lack reproducibility, largely due to the different analysis settings. This is particularly important for the development of imaging biomarkers. The aim of this work was to evaluate the reproducibility of our recent study regarding the functional connectivity of the basal ganglia network in early Parkinson's disease (PD) (Szewczyk-Krolikowski et al., 2014). In particular, we systematically analysed the influence of two rfMRI analysis steps on the results: the individual cleaning (artefact removal) of fMRI data and the choice of the set of independent components (template) used for dual regression. Our experience suggests that the use of a cleaning approach based on single-subject independent component analysis, which removes non neural-related sources of inter-individual variability, can help to increase the reproducibility of clinical findings. A template generated using an independent set of healthy controls is recommended for studies where the aim is to detect differences from a "healthy" brain, rather than an "average" template, derived from an equal number of patients and controls. While, exploratory analyses (e.g. testing multiple resting state networks) should be used to formulate new hypotheses, careful validation is necessary before promising findings can be translated into useful biomarkers.

Keywords: Artefact removal; Basal ganglia network; Dual regression; Functional connectivity; Parkinson's disease; Resting state functional magnetic resonance imaging (rfMRI).

Fig. 1

A) Temporal SNR of data obtained with different cleaning options. B) Spatial correlation between single-subject BGN maps, using the same template for dual regression as in Szewczyk-Krolikowski et al. (2014), obtained using data cleaned with different options and the corresponding map obtained using manually cleaned data (i.e. after removing the contribution of manually selected artefactual components after single-subject ICA).

Fig. 2

Average P.E. in the basal ganglia ROIs extracted from single-subject BGN maps obtained from data cleaned with different options. *Significant between-group differences surviving Bonferroni correction across structures.

Fig. 3

Voxel-wise between-group differences in the BGN (PD < HC) using different automated cleaning approaches on the full sample (30HC vs 59 PD). Each map is independently corrected for multiple comparisons using the TFCE approach. No significant differences were found using the opposite contrast (PD > HC).

Fig. 4

Average P.E. in the basal ganglia ROIs extracted from single-subject BGN maps obtained using different templates for dual regression. *Significant between-group differences surviving Bonferroni correction across structures.

Fig. 5

Voxel-wise between-group differences in the BGN (PD < HC) using different templates for dual regression. Each map is independently corrected for multiple comparisons using the TFCE approach. No significant differences were found using the opposite contrast (PD > HC).

Fig. 6

Similarity (spatial correlation of t-maps) and overlap (Dice index on thresholded t-maps) among voxel-wise analyses in the BGN using different templates for dual regression when A) changing subjects, using a subset of components including BGN and noise (Fig. 4. a–c, a–e, c–e); B) changing the set of components (subset vs all), but using the same subjects to generate the template (Fig. 4. a–b, c–d, e–f); C) changing subjects, using all components (Fig. 4. b–d, b–f, d–f); D) changing subjects and set of components (Fig. 4. other pairs).