Continuous carry-over designs for fMRI

Abstract

This paper describes continuous carry-over fMRI experiments. In these studies, stimuli are presented in an unbroken, sequential manner, and can be used to estimate simultaneously the mean difference in neural activity between stimuli as well as the effect of one stimulus upon another (carry-over effects). Neural adaptation, which has been the basis of many recent fMRI studies, is shown to be a specific form of carry-over effect. With this approach, the adapting effects of stimuli may be studied in a continuous sequence, as opposed to within isolated pairs or blocks. Additionally, the average, direct effect of a stimulus upon neural response can form the basis of a simultaneously obtained distributed pattern analysis, allowing comparison of neural population coding on focal (within voxel) and distributed (across voxel) spatial scales. These studies are ideally conducted with serially balanced sequences, in which every stimulus precedes and follows every other stimulus. While m-sequences can provide this stimulus order, the type 1 index 1 sequence of Finney and Outhwaite may be used in fMRI studies for those experimental designs for which an m-sequence solution does not exist. Continuous carry-over designs with serially balanced sequences are argued to be particularly well suited to the characterization of "similarity spaces," in which the perceptual similarity of stimuli is related to the structure of neural representation both within and across voxels. These concepts are illustrated with a worked example involving the neural representation of color. It is shown that data from a single scanning session are sufficient to detect direct and carry-over effects, as well as demonstrate the correspondence of the similarity structure of distributed patterns of neural firing and the perceptual similarity of a set of colors.

Figure 1. An example carry-over matrix (ϕ), and a restricted basis-set of matrices

The grayscale value within each cell of the matrix reflects the proportional size of the carry-over effect for the sequential transition between two stimuli. The labels given to the matrices correspond to those used in the text, with λ modeling a symmetric carry-over effect (for linear, quadratic and cubic components), ρ representing an asymmetric effect (termed bias in the text), and κ modeling the carry-over effect of seeing the same stimulus twice in succession. The ellipsis indicates that many other matrices, modeling other types of carry-over effects, could be added to this set.

Figure 2. Examples of stimulus spaces and their implied similarity matrices

A) On the left, an example set of 8 stimuli consisting of a bar of light rotated in 22.5° increments. On the right is the similarity matrix for these stimuli. B) A contrived semantic similarity space for birds. If subjects were asked to provide ratings of the similarity of pairs of different birds, a representational space such as depicted on the left of the figure might emerge, in which some groups of birds are found to be more similar than others. This might represent, for example, the 2-dimensional projection of the results of a multi-dimensional scaling analysis conducted upon the similarity ratings (Steyvers, 2002). C) A stimulus space that consists of X and Y position within a city-block grid. Unlike a Euclidean space, the distance between two points is defined by |x₁ − x₂| + |y₁ − y₂|, termed Manhattan or city-block geometry. Manhattan geometry is also encountered in similarity spaces for stimuli with very separable dimensions (e.g., size and brightness; Attneave, 1950). D) The Optical Society of America (OSA), committee for Uniform Color Scales (UCS) color space. The similarity matrix assumes equal perceptual saliency of steps along any of the three axes and a Euclidean relationship between the axes.

Figure 3. Design matrices used in analysis of focal and distributed neural responses

Time runs along the vertical axis from top to bottom. On the left is shown the serially balanced sequence used in the experiment (and presented in Appendix B), with each label replaced with the corresponding stimulus color. Each column of the design matrices represents a particular covariate over time. The “focal” design matrix was used in the data analysis shown in Figure 4. The “distributed” design matrix, which modeled each presentation of a given color independent of context, was used in the analysis of distributed patterns of neural response shown in Figure 5B. In the example study, the sequence was presented twice for a total 40 minute duration, but only one half of the identical, replicated design matrix is shown.

Figure 4. Focal effects

A) The main effect of stimuli vs. null trials (and the added effect of stimuli following null-trials), presented on the inflated cortical surface. Points of activity correspond to F values greater than 30. The regions of activity seen here provided the signal used for the regional analyses presented in the subsequent panels. B) Carry-over adaptation effects within the regions defined in panel A. For each region, the t-value for the linear carry-over adaptation effect across all colors was obtained, and the fitted adaptation response plotted. The x-axis is the distance between adjacent stimuli in the color space, and the y-axis is the % fMRI signal change, relative to the average response to any color that followed a different color. Also shown is the response to a color that followed an identical color (step size of zero), modeled with a separate, orthogonal covariate (κ). In blue is the fitted response of a region in the left, dorsolateral pre-frontal cortex identified by a whole-brain F-test of the linear, quadratic and cubic symmetric carry-over (adaptation) covariates. C) For each visual area, the loading upon the linear, L-direction, asymmetric carry-over (bias) covariate is shown, expressed as % fMRI signal change. Negative values indicate a contrastive effect, in that (e.g.) the response to a darker stimulus was enhanced when it was preceded by a lighter stimulus. Contrast effects were found in primary visual cortex, and a trend towards the opposite, assimilative effect, was seen in area MT+. D) Region direct effects. For each visual area, the modulation of neural response as a function of the color of the stimulus was modeled for the three cardinal color directions, and this set of covariates assessed with an F-test. Five visual areas showed a significant modulation of response by direct effects. For each area, the tri-plot indicates the magnitude and direction of modulation of response as a function of position along the color axes. Each tri-plot expresses the modulation as a proportion of a 0.2% signal change, with the positive direction being increases in signal as one moves in color space away from the lightest, yellowest, reddest stimulus (i₁)

Figure 5. Distributed neural similarity

A) Displayed on T1 axial slices, voxels located within posterior visual areas with the highest 1000 F-values for the main effect of stimulus presentation (μ and λ₀). B) Calculation of the neural similarity matrix. The vector of β values obtained across voxels for each color were compared by correlation. The resulting r value provides one cell of the neural similarity matrix. Only the lower triangle (omitting the diagonal) of this diagonally symmetric matrix is shown. C) Shown is the perceptual similarity matrix for color space (derived from Figure 2D), as well as decompositions of this matrix along the three color axes. The correlation between the neural similarity matrix shown in panel B with the perceptual similarity matrices is indicated. The G and J color directions were evaluated conjointly.

Appendix Figure 1. The efficiency ratio (obtained through simulation) of two covariates derived from a serially-balanced experimental design

In all panels, solid indicates E(τ) and dashed indicates E(λ₁), as defined by the similarity matrix from Figure 2D. A) The effect of null-trial duration upon relative power. Selectively increasing the duration of the null-trials improves efficiency of covariates that model direct effects. B) Increasing the number of labels assigned to null-trials does not improve efficiency. C) Increasing the duration of stimulus presentation theoretically enhances the efficiency of covariates for both effects in a serially balanced design. D) Searches across permuted labeling of stimuli within a particular serially balanced sequence leads to a modest improvement in efficiency for detection of adaptation effects, but no change in the efficiency of covariates modeling direct effects.