Neuronal responses to texture-defined form in macaque visual area V2

Abstract

Human and macaque observers can detect and discriminate visual forms defined by differences in texture. The neurophysiological correlates of visual texture perception are not well understood and have not been studied extensively at the single-neuron level in the primate brain. We used a novel family of texture patterns to measure the selectivity of neurons in extrastriate cortical area V2 of the macaque (Macaca nemestrina, Macaca fascicularis) for the orientation of texture-defined form, and to distinguish responses to luminance- and texture-defined form. Most V2 cells were selective for the orientation of luminance-defined form; they signaled the orientation of the component gratings that made up the texture patterns but not the overall pattern orientation. In some cells, these luminance responses were modulated by the direction or orientation of the texture envelope, suggesting an interaction of luminance and texture signals. We found little evidence for a "cue-invariant" representation in monkey V2. Few cells showed selectivity for the orientation of texture-defined form; they signaled the orientation of the texture patterns and not that of the component gratings. Small datasets recorded in monkey V1 and cat area 18 showed qualitatively similar patterns of results. Consistent with human functional imaging studies, our findings suggest that signals related to texture-defined form in primate cortex are most salient in areas downstream of V2. V2 may still provide the foundation for texture perception, through the interaction of luminance- and texture-based signals.

Figure 1.

Stimulus composition. Texture patterns (herringbones) were composed by periodic modulation of two orthogonal static luminance gratings (carriers) by two drifting gratings (modulators). A fixed orientation difference of ±45° separated modulator and carrier patterns, and was used to distinguish neuronal responses to each. A, D, Low spatial frequency drifting gratings used to modulate the carrier patterns (modulators M and its inverse −M). B, C, High spatial frequency static gratings used as carrier patterns. To produce the texture pattern, each carrier was multiplied by a modulator (carrier 1 with modulator M; carrier 2 with modulator −M). E, F, The resulting contrast-modulated carrier patterns. G, The final herringbone texture pattern was generated by combining the patterns in E and F additively; the modulator orientation (here vertical) defined the orientation of the texture-defined form.

Figure 2.

Response predictions. Tuning of a hypothetical V2 neuron to gratings and herringbones, plotted on polar coordinates; modulator direction is represented along the circumference and response magnitude along the radius. A, Tuning for grating orientation. In this example, the neuron prefers vertical gratings moving right or left (black arrows on the preferred stimulus icon, right). B–D, Possible herringbone response patterns, shown as a function of modulator direction. B, First-order prediction. A neuron selective for luminance-defined form would prefer herringbone patterns containing carriers that match its optimal grating (vertical, shown in white ellipses on the preferred stimulus icons, below). This occurs twice for each carrier, ±45° away from the modulator, resulting in a four-lobed tuning curve. C, Modulated by motion prediction. A neuron selective for both the orientation of luminance-defined form and the direction of second-order motion would prefer herringbone patterns containing vertical carriers (same white ellipses as in B) but only—for example—when the modulator moves rightward. Preferred modulator directions are indicated by black arrows (up/right and down/right); nonpreferred modulator directions are indicated by light gray arrows (up/left and down/left). Predicted herringbone tuning is aligned with the four-lobed linear prediction, but only two adjacent lobes are evident. D, Modulated by form prediction. A neuron selective for the orientation of luminance-defined form and the spatial structure of the second-order envelope would prefer herringbone patterns containing vertical carriers (same white ellipses as in B, C), but only when the flanking horizontal elements (black ellipses) form a particular spatial configuration: for example, a “z” shape configuration (stimulus icon with black arrows) and not when they form an “s” shape configuration (icon with gray arrows). E, The second-order cue-invariant prediction. A neuron selective for the orientation of texture-defined form would prefer herringbone patterns containing a vertical modulator that matches its optimal grating, thus responding similarly to luminance- and texture-defined form.

Figure 3.

Responses of example V2 neurons to texture patterns. Shown are data from five neurons recorded in monkey V2. For each, we plot the measured grating tuning (column 1), the first-order linear response prediction (column 2) (compare Fig. 2B), tuning for carrier-exchange controls (column 3), and herringbone tuning (column 4). The tuning curves for each neuron were aligned such that its preferred grating orientation was rotated to 0°. A, Responses of a first-order neuron. Herringbone tuning was four-lobed and matched the shape of the linear prediction and carrier-exchange controls. B, Responses of a neuron that was modulated by motion. Herringbone tuning was directional, and only two adjacent tuning lobes were evident. C, Responses of a neuron that was modulated by form. Herringbone tuning had a contextual signature, and only two opposite lobes were evident. D, Responses of a second-order cue-invariant neuron; herringbone and grating tuning curves were similar in shape. E, Responses of a second-order cue-orthogonal neuron; herringbone and grating tuning curves were orthogonal. Baseline firing rates are represented by gray circles; response magnitudes are indicated by numbers on the outer circles of the polar plots. Column 5 shows the log likelihood (probability) of each of the possible response predictions accounting for the measured herringbone tuning. Log likelihoods were normalized separately for each cell, and transformed to a scale of 0 to 1, from the least to the most likely (see Materials and Methods). Labels for the response predictions (abscissa) are abbreviated as follows: first-order (1°), modulated by motion (1° motion), modulated by form (1° form), second-order cue-invariant (2° cue-invariant), and second-order cue-orthogonal (2° cue-orthogonal).

Figure 4.

Distribution of selectivity for luminance- and texture-defined form in V2. Each point represents a single neuron (N = 128); its position relative to each edge of the triplot represents how well each family of response predictions (first-order, second-order, and an “intermediate” family of modulated responses: modulated by motion or form) can account for the observed herringbone tuning (for more detail, see Materials and Methods). The points are color-coded by the log likelihood of the best prediction (compare Fig. 3, column 5); darker shades represent higher likelihoods. Most neurons were best described by the first-order prediction, some cells fell into the intermediate category, whereas only a handful of cells were best described by the second-order prediction. The second-order neuron marked as an open circle is the only cue-invariant neuron we recorded in V2 (Fig. 3D).

Figure 5.

Distribution of selectivity for luminance- and texture-defined form in monkey V1. Control V1 data (N = 26) are represented in the same way as in Figure 4. Most V1 neurons were selective for luminance-defined not texture-defined form. The distribution was qualitatively similar to that recorded in V2 (Fig. 4).

Figure 6.

Responses to carrier spatial frequencies beyond and within the resolution limit. Shown are data from an example V2 neuron. We show the tuning of the neuron for grating spatial frequency (column 1; triangles marked “m” and “c” indicate the modulator and carrier spatial frequencies used), tuning for carrier-exchange controls (column 2), and the herringbone patterns (column 3). A, Patterns composed of carrier spatial frequencies beyond the resolution limit failed to elicit visual responses in this V2 neuron (gray circles in polar plots show baseline firing; numbers on outer circles indicate response magnitude). B, Patterns composed of carrier spatial frequencies within the resolution limit produced orientation-tuned responses to both stimuli. This response pattern was true for most neurons recorded in V2. For a subset of neurons (N = 33), we measured herringbone tuning under both parameter conditions and examined distributions of relative log likelihoods (column 4). With carriers beyond the resolution limit, most neurons responded weakly and were not statistically classified (points near origin in top row, column 4); other neurons were selective for luminance-defined form. With carriers within the resolution limit, responses were well tuned, and most neurons were strongly classified as selective for luminance-defined form.

Figure 7.

Responses to full and half herringbone patterns. Shown are data for three example V2 neurons (shown in Fig. 3) to full herringbone patterns (as in Fig. 1G) and the two half herringbone patterns (contrast-modulated gratings, as in Fig. 1E,F). For each neuron, we show the measured grating tuning (column 1), the linear response prediction (column 2), tuning for carrier-exchange controls (column 3), and the full herringbone patterns (column 4, see stimulus icon). We also show tuning for each half pattern (half 1 and half 2, columns 5 and 6; see stimulus icons above). A, Responses of a first-order neuron. Tuning for full herringbone patterns was four-lobed; tuning for each half pattern was two-lobed and aligned with the linear prediction, reflecting the response to each carrier pattern present. B, Responses of a modulated by motion neuron. Full herringbone tuning was directional, with only two of the four tuning lobes evident; tuning for each half pattern was also directional, with only one lobe present, reflecting the response to each carrier. C, Responses of a second-order cue-orthogonal neuron. Full herringbone tuning did not match the four-lobed linear prediction; tuning for each half pattern was also identical in shape, indicating that it was cue-orthogonal for both stimulus types. D, Distribution of relative log likelihoods for V2 neurons tested with full herringbone patterns (N = 25). E, Comparison distribution for the same V2 neurons when tested with half herringbone patterns. We computed likelihoods from the responses to each half separately and plotted the average normalized likelihood for both halves. The distribution was qualitatively similar, with most neurons classified as first order, and a few as “second-order cue-orthogonal.”

Figure 8.

Direction selectivity for gratings and herringbones. A, B, Data for two example V2 neurons with directional herringbone responses, where only two adjacent tuning lobes were evident. For each, we show the tuning for grating direction (column 1) from which we generated the nondirectional grating tuning curve (column 2) that formed the basis of our linear response prediction (column 3). We also show the measured tuning for herringbone direction (column 4). A, Responses of a neuron that was strongly direction selective for gratings, with a high direction selectivity index (DSI = 0.83). It was also strongly direction selective for herringbones, with a high herringbone direction selectivity index (DSI_H = 1) (see text). B, Responses of a neuron that was weakly direction selective to gratings (DSI = 0.32) but was nevertheless strongly direction selective to herringbones (DSI_H = 1).

Figure 9.

Relationship between grating and herringbone direction selectivity. For all V2 neurons (N = 128), we plot DSI_H against grating DSI. There was a modest but significant correlation (r² = 0.0967; p = 0.0004), suggesting that directional grating responses may account for some of the directional responses to second-order motion in texture patterns. Points are color-coded by the log likelihood of the modulated by motion prediction; the darker shades represent higher likelihoods.

Figure 10.

Surround suppression and herringbone contextual selectivity. A, B, Data for two example V2 neurons that showed contextual herringbone responses with two prominent tuning lobes that were opposite. For each, we show tuning for grating size (column 1), for grating orientation (column 2), for herringbone orientation (column 4), and the first-order linear prediction (column 3). A, Responses of a neuron that was strongly suppressed by stimuli that extended beyond the receptive field center, resulting in a high surround suppression index (SSI = 1, above the plot; gray line indicates baseline activity; black line is a descriptive model fit). The neuron showed strong contextual tuning, with a high herringbone contextual selectivity index (CSI_H = 0.98) (see text). B, Responses of a neuron that was also strongly surround suppressed (SSI = 1) but showed only weak contextual herringbone tuning (CSI_H = 0.14).

Figure 11.

Relationship between surround suppression and herringbone contextual selectivity. For all V2 neurons (N = 128), we plot herringbone contextual selectivity index (CSI_H) against grating SSI. There was no significant correlation between these indices (r² = 0.0006; p = 0.7775), suggesting that the overall strength of surround suppression could not easily account for neuronal preferences for particular spatial configurations of carrier elements (“z” vs “s”). Points are color-coded by the log likelihood of the modulated by form prediction; the darker shades represent higher likelihoods.