Orientation decoding in human visual cortex: new insights from an unbiased perspective

Abstract

The development of multivariate pattern analysis or brain "decoding" methods has substantially altered the field of fMRI research. Although these methods are highly sensitive to whether or not decodable information exists, the information they discover and make use of for decoding is often concealed within complex patterns of activation. This opacity of interpretation is embodied in influential studies showing that the orientation of visual gratings can be decoded from brain activity in human visual cortex with fMRI. Although these studies provided a compelling demonstration of the power of these methods, their findings were somewhat mysterious as the scanning resolution was insufficient to resolve orientation columns, i.e., orientation information should not have been accessible. Two theories have been put forth to account for this result, the hyperacuity account and the biased map account, both of which assume that small biases in fMRI voxels are the source of decodable information. In the present study, we use Hubel and Wiesel's (1972) classic ice-cube model of visual cortex to show that the orientation of gratings can be decoded from an unbiased representation. In our analysis, we identify patterns of activity elicited by the edges of the stimulus as the source of the decodable information. Furthermore, these activation patterns masquerade as a radial bias, a key element of the biased map account. This classic model thus sheds new light on the mystery behind orientation decoding by unveiling a new source of decodable information.

Keywords: fMRI decoding; hyperacuity; multivariate pattern analysis; orientation columns; visual cortex.

Figure 1.

The PCM. A, Sixty-four phase-shifted stimulus exemplars. B, Oriented filters acting as simple cells encoding orientation are convolved with the stimulus exemplars. C, The response of each filter to each exemplar is squared, and then the exemplar responses are summed to make an activity map for each orientation filter. D, The filters' response maps are summed into a single pattern response map, in which each pixel is a “cube” that has a perfectly balanced contribution from each orientation channel, representing an unbiased fMRI voxel.

Figure 2.

Decoding results. A–D, Correlation classifier DSMs for the four sets of grating stimuli. Rows and columns are stimulus orientation. Individual entries in the DSM are correlation distances (i.e., the decodability) between pairs of oriented gratings. Inset, Example stimuli. A, Square-wave grating with a small inner radius edge. B, Sine-wave grating with a small inner radius edge. C, Sine-wave grating with a large inner radius edge. D, Sine-wave grating with a small inner radius edge and heavy blur function. Note the scale change in dissimilarity for A–C DSMs to the DSM for D.

Figure 3.

Edge related activity. A, Orientation filter-response patterns for a vertical stimulus. The oriented filters are shown at the top of each column. Vertically oriented exemplars from each stimulus set are shown at the beginning of the rows. Pattern responses from the filters to the vertical exemplar from each sets shown as images. Pattern responses are scaled individually from 0 to 1. B–E, DSM of the differential response between oriented stimuli. Rows and columns are exemplar orientations. Each entry in the DSM is the absolute difference in the model response to a pair of exemplars. The four DSMs were normalized collectively to range from 0 to 1 for the purposes of comparing responses across stimulus sets. Example stimuli are shown in the insets. B, Square-wave grating with a small inner radius edge. C, Sine-wave grating with a small inner radius edge. D, Sine-wave grating with a large inner radius edge. E, Sine-wave grating with a small inner radius edge and heavy blur function.

Figure 4.

Model activity mimics a radial bias. A, Top, Orientation exemplars from the sine-wave stimulus set with a large inner edge radius. Middle, PCM's pattern response. Bottom, Log amplitude of the 2D Fourier analysis on the model response. Spatial frequency is lowest in the center of the image and increases from the center. The highlighted central region indicates frequencies below the Nyquist sampling rate. B–E, PCM activity as a function of polar angle. The x-axis is a double wedge region of the PCM's pattern response (diagrammatically shown below the axis). The wedge has a width of 45° and its angular position shifts stepwise in 22.5° steps. The y-axis is the mean PCM activation within the defined region. The lines are different stimulus orientations. B, Square-wave grating with a small inner radius edge. C, Sine-wave grating with a small inner radius edge. D, Sine-wave grating with a large inner radius edge. E, Sine-wave grating with a small inner radius edge and heavy blur function.

Figure 5.

Edge related activity mimics a radial bias. A, Mean activation of the PCM as a function of polar angle for 3 radiuses: the inner edge (left), the center of the annulus (middle), and the outer edge (right). The image in the inset of the legends graphically shows the radius. The x-axis is a defined double wedge region with a width of 45° that shifts stepwise in 22.5° steps (diagrammatically shown below the axis). The y-axis is the mean PCM activation within the defined region. The lines are different stimulus orientations. Activity was normalized collectively over the three radius conditions (inner edge, middle of the annulus, and outer edge) to range from 0 to 1 for the purposes of comparison. B–D, Mean activation of the PCM as a function of polar angle for two radiuses: the center of the annulus (top), and the outer edge (bottom) for the square-wave stimulus (B), the sine-wave grating stimuli with a small inner radius (C), and the heavily smoothed sine-wave grating stimuli with a large inner radius (D).

Figure 6.

Orientation coding and edge activity. A, The response to oriented gratings by filters with different levels of OD. Shown are four oriented grating exemplars with 0°, 45°, 90°, and 135° orientations. To the right of the exemplars is the response of filters with varying levels of OD. The orientation disparity of the filters increases left to right with 0°, 22.5°, 45°, 67.5°, and 90° offsets (top row). Below each filter the filter's pattern response. The dashed line placed over the pattern response indicates the orientation of the peak response to the edge. B, Summed response of the positive (clockwise) and negative (counterclockwise) OD filter responses a vertical grating pattern. C, Graphical depiction of interaction between filter disparity and the stimulus edge. Shown is the upper right quadrant of a vertically sine-wave stimulus. Semitransparent filters with varying OD are superimposed over the stimulus in the location corresponding to the peak edge response (indicated by the dashed line) for the filter. From the top to the bottom, plots show filters with increasing OD. D–G, Individual filters with varying OD contributions to the outer edge response. Shown are area plots showing the cumulative response of the filters to the outer edge response to a vertical grating. Inset, The grating exemplar from each stimulus set. The x-axis is a double wedge with a width of 45° region constrained to the outer edge that shifts stepwise in 22.5° steps (diagrammatically shown below the axis). The y-axis is the filter's response within the wedge. The filled regions are the response of individual filters. Filter orientation is described relative to the stimulus orientation. D, Sine-wave grating with a small inner radius edge. E, Square-wave grating with a small inner radius edge. F, Sine-wave grating with a large inner radius edge. G, Sine-wave grating with a small inner radius edge and heavy blur function.

Figure 7.

Decoding spirals. A, Spiral sense exemplars. B, Multiscale PCM's spatial pattern response output difference between spiral sense for the six different spatial frequency channels (SFC): 2.50, 1.25, 0.8250, 0.6250, 0.5000, and 0.4250 cycles/°. C, Mean activation as a function of polar angle for the different spatial frequency channels. The x-axis is a double wedge region with a width of 90° that shifts stepwise in 22.5° steps (diagrammatically shown below the axis). The y-axis is the mean layer activation within the defined region for the spatial frequency channel. D, Spatial difference between senses after combining the six multiscale PCM spatial frequency channels. E, Mean activation as a function of polar angle. F, Mean activation as a function of polar angle for four radiuses: the inner edge (circles), the outer edge (diamonds), and two radiuses inside the stimulus (squares and crosses).