Edge-Related Activity Is Not Necessary to Explain Orientation Decoding in Human Visual Cortex

Abstract

Multivariate pattern analysis is a powerful technique; however, a significant theoretical limitation in neuroscience is the ambiguity in interpreting the source of decodable information used by classifiers. This is exemplified by the continued controversy over the source of orientation decoding from fMRI responses in human V1. Recently Carlson (2014) identified a potential source of decodable information by modeling voxel responses based on the Hubel and Wiesel (1972) ice-cube model of visual cortex. The model revealed that activity associated with the edges of gratings covaries with orientation and could potentially be used to discriminate orientation. Here we empirically evaluate whether "edge-related activity" underlies orientation decoding from patterns of BOLD response in human V1. First, we systematically mapped classifier performance as a function of stimulus location using population receptive field modeling to isolate each voxel's overlap with a large annular grating stimulus. Orientation was decodable across the stimulus; however, peak decoding performance occurred for voxels with receptive fields closer to the fovea and overlapping with the inner edge. Critically, we did not observe the expected second peak in decoding performance at the outer stimulus edge as predicted by the edge account. Second, we evaluated whether voxels that contribute most to classifier performance have receptive fields that cluster in cortical regions corresponding to the retinotopic location of the stimulus edge. Instead, we find the distribution of highly weighted voxels to be approximately random, with a modest bias toward more foveal voxels. Our results demonstrate that edge-related activity is likely not necessary for orientation decoding.

Significance statement: A significant theoretical limitation of multivariate pattern analysis in neuroscience is the ambiguity in interpreting the source of decodable information used by classifiers. For example, orientation can be decoded from BOLD activation patterns in human V1, even though orientation columns are at a finer spatial scale than 3T fMRI. Consequently, the source of decodable information remains controversial. Here we test the proposal that information related to the stimulus edges underlies orientation decoding. We map voxel population receptive fields in V1 and evaluate orientation decoding performance as a function of stimulus location in retinotopic cortex. We find orientation is decodable from voxels whose receptive fields do not overlap with the stimulus edges, suggesting edge-related activity does not substantially drive orientation decoding.

Keywords: fMRI decoding; hyperacuity; multivariate pattern analysis; orientation columns; population receptive field mapping; visual cortex.

Figure 1.

Stimuli and fMRI experimental design for (A) the orientation experiment and (B) the pRF mapping sessions. C, Predicted pattern of data if edge-related activity substantially contributes to orientation decoding. Ci, When all voxels are available for classification, voxels with pRFs located at the stimulus edges should contribute more to orientation decoding than voxels with pRFs corresponding to the middle of the stimulus. Cii, When voxels are binned by pRF eccentricity before classification, higher classification performance is expected for bins with voxel eccentricities centered near the stimulus edges (at 3.5° and 9.5°).

Figure 2.

Orientation decoding accuracy, averaged across all 4 subjects and 15 orientation pairs. Chance performance is 50%. Error bars indicate between-subjects SEM. A, Decoding accuracy for V1 and the V1 ROI defined as the subset of V1 voxels responsive to the stimulus location (for voxel counts, see Table 1) used in all subsequent analyses. B, Orientation decoding as a function of the stimulus region (stimulus edges or middle of grating). Because of the cortical magnification factor, the number of voxels in each ROI falls off systematically from the inner to outer stimulus edge. To equate ROI size, 50 voxels were resampled from each ROI 100 times to compare decoding performance for equal ROI sizes (dark red bars).

Figure 3.

Decoding tuning curves. Orientation decoding by binned pRF values (bin size = 50 voxels); individual data shown for all four subjects. x-axis indicates the maximum value of each parameter per bin. Leftover ROI voxels not fitting into the highest 50 voxel bin are discarded from this analysis. Chance classification performance is 50%. Decoding accuracy is shown as a function of voxel population receptive field parameters. A, Eccentricity. Shaded region represents stimulus area. B, Polar angle. C, Receptive field size (half-width at half-height). D, Goodness of fit for the pRF Gaussian-hdr model. Best fitting cubic polynomials (A–C) and straight lines (D) are plotted, fitted to the individual data with weighted least-squares. Individual outlier bins with maximum eccentricities exceeding 13° (A) or pRF sizes outside the range 0.5°–4° HWHM (C) are discarded from the curve fitting. Error bars indicate within-subjects SEM of decoding performance across the 15 unique orientation pairs for each 50 voxel bin.

Figure 4.

Distribution of voxels driving orientation classification performance, including all voxels from the 4 subjects. A, Decoding performance as a function of raw classifier weight (left) and transformed classifier pattern (right). Bin size = 50 voxels. Weights and patterns are derived from prior classification with all voxels. Voxels not fitting into the final (low weight) full-sized bin are discarded from the analysis. B, Proportion of voxels with pRFs overlapping with the stimulus edges, defined as a function of pRF eccentricity and half-width at half-height. Voxels with transformed patterns ≥2 SDs above the mean are classed as high performing voxels for classification. C–E, Histograms comparing the distributions of polar angle, pRF size (HWHM), or pRF goodness of fit of all voxels (light gray) versus those whose contribution to classifier performance was ≥2 SDs above the mean for at least 1 of the 15 orientation classification pairs (dark gray). F, Center of each voxel's pRF. Color represents relative contribution to classification performance across all 15 orientation classification pairs. Gray annulus indicates the stimulus location. G, Histogram comparing the distributions of eccentricity for all voxels (light gray) versus the eccentricity of voxels whose contribution to classifier performance was ≥2 SDs above the mean for at least 1 of the 15 orientation classification pairs (dark gray). Red lines indicate the stimulus edges at 3.5° and 9.5°.

Figure 5.

Distribution of voxels with population receptive fields between 1.5–2° half-width at half maximum (HWHM). All voxels in this pRF size range are included for all subjects. Voxels with transformed patterns ≥2 standard deviations above the mean are classed as high performing voxels for orientation classification. A, Histogram comparing the distributions of eccentricity for all voxels (light grey) within the selected pRF size range (1.5–2° HWHM) versus the eccentricity of voxels in this range whose contribution to classifier performance was ≥2 standard deviations above the mean for at least one of the 15 orientation classification pairs (dark grey). Red lines mark the stimulus edges at 3.5 and 9.5°. B, Proportion of voxels in the selected size range (1.5–2° HWHM) with population receptive fields overlapping with the stimulus edges, defined as a function of pRF eccentricity and half width at half height.