Long-term test-retest reliability of functional MRI in a classification learning task

Abstract

Functional MRI is widely used for imaging the neural correlates of psychological processes and how these brain processes change with learning, development and neuropsychiatric disorder. In order to interpret changes in imaging signals over time, for example, in patient studies, the long-term reliability of fMRI must first be established. Here, eight healthy adult subjects were scanned on two sessions, 1 year apart, while performing a classification learning task known to activate frontostriatal circuitry. We show that behavioral performance and frontostriatal activation were highly concordant at a group level at both time-points. Furthermore, intra-class correlation coefficients (ICCs), which index the degree of correlation between subjects at different time-points, were high for behavior and for functional activation. ICC was significantly higher within the network recruited by learning than outside that network. We conclude that fMRI can have high long-term test-retest reliability, making it suitable as a biomarker for brain development and neurodegeneration.

Fig. 1

Scanning design for probabilistic classification learning (PCL) and baseline trials. (a) On each occasion (session), subjects performed 2 scans, each consisting of 10 cycles of 5 PCL trials and 3 baseline trials (80 trials total per scan). (b) On each weather prediction trial, a stimulus was presented, comprising 1 to 3 cards, at randomized locations, for up to 4 s. Within that time, the subject responded with left button press (sun) or right button press (rain). Feedback (“sunshine” or “rain”) was presented after button press for the remainder of the 4-s window. Intertrial interval was 0.5 s. (c) Baseline trials controlled for visual stimulation, button press and computer response to button press. A standard card was always presented in all 3 positions along with the instruction to press (subjects always pressed the right-hand key for these trials). (d) Four cards were used for PCL trials in first and second sessions. Assignment of cards to subjects was pseudo-random.

Fig. 2

Behavioral data from first and second scanning sessions. (a) Mean accuracy for the subjects improved significantly across scans within each session ( P < 0.0001), but there was no significant difference in accuracy between sessions. For the 8 subjects, mean accuracy for session 1 was significantly correlated with mean accuracy at session 2 (ICC = 0.85, P < 0.01), and this pattern was also evident for a between-session comparison of scan 1 (b) and scan 2 (c).

Fig. 3

Learning in both sessions is associated with robust activation of midbrain and frontostriatal regions. For each session, a random effects analysis is run with contrast images (PCL trials minus baseline trials) for 8 subjects. The activations shown are significant after correction for multiple comparisons at the cluster level P < 0.01, voxel level threshold is z > 2.3. In both sessions, there is significant activation of midbrain, striatal, orbital, lateral and medial frontal cortex, as well extra-striate visual areas.

Fig. 4

High test–retest reliability of fMRI signals within frontostriatal areas. (a) ICC values exceeding 0.5 are shown on a voxel-by-voxel basis within regions which were significantly activated for PCL vs. baseline for session 1 OR session 2 (inclusively). Voxels within midbrain, striatal, orbital, dorsolateral and medial frontal cortex show high ICC. (b) Illustrative signal plots within key regions of interest (ROIs) of this network. The ROIs were based on prior neuropsychological and neuroimaging research which has implicated midbrain, striatal and frontal foci (Aron et al., 2004; Knowlton et al., 1996; Moody et al., 2004; Poldrack et al., 2001; Seger and Cincotta, 2005; Shohamy et al., 2004). Mean signal within a sphere of 4 mm radius was extracted for each subject and each session. The center of the sphere is demarcated by MNI coordinates [x y z]. (c) Panel showing each of the 9 ROIs on axial slices.

Fig. 5

Reliability within the probabilistic classification learning (PCL) network is significantly higher than for brain regions outside that network. Relative frequency histogram of ICC values for PCL network and area outside PCL network, exPCL (excluding zero values and negative values in both cases). Values within the PCL network are significantly higher than for those in exPCL (Chi-square test for difference between distributions, P < 0.0001); confirming that test– retest reliability is greater in areas important for task performance.