The role of temporal cortical areas in perceptual organization

Abstract

The visual areas of the temporal lobe of the primate are thought to be essential for the representation of visual objects. To examine the role of these areas in the visual awareness of a stimulus, we recorded the activity of single neurons in monkeys trained to report their percepts when viewing ambiguous stimuli. Visual ambiguity was induced by presenting incongruent images to the two eyes, a stimulation condition known to instigate binocular rivalry, during which one image is seen at a given time while the other is perceptually suppressed. Previous recordings in areas V1, V2, V4, and MT of monkeys experiencing binocular rivalry showed that only a small proportion of striate and early extrastriate neurons discharge exclusively when the driving stimulus is seen. In contrast, the activity of almost all neurons in the inferior temporal cortex and the visual areas of the cortex of superior temporal sulcus was found to be contingent upon the perceptual dominance of an effective visual stimulus. These areas thus appear to represent a stage of processing beyond the resolution of ambiguities--and thus beyond the processes of perceptual grouping and image segmentation--where neural activity reflects the brain's internal view of objects, rather than the effects of the retinal stimulus on cells encoding simple visual features or shape primitives.

Figure 1

Example stimuli used during the experiments. Stimuli consisted of geometrical sunburst patterns (left-objects), images of animate objects (right-objects), and physical blends of images that were used to mimic piecemeal rivalry (mixed-objects). The monkeys were trained to pull the left lever whenever the left-objects were visible, the right-lever whenever right-objects were visible, and neither lever when mixed-objects were visible.

Figure 2

Behavioral verification of monkey’s performance during rivalry. (A) Each pair of images depicts a stimulus condition, wherein the image of the face remained unchanged while that of the sunburst was blurred, to various degrees, by lowpass filtering. Filtering was achieved by multiplying the amplitudes of forward Fourier transformed images by an exponential gain and then converting back to the space domain. The lowpass cutoffs shown below each image refer to the frequency at which the exponential filter was equal to 1/e. (B) Predominance of a stimulus as function of spatial frequency bandwidth. Predominance is defined here as T_sunburst/(T_sunburst + T_face), where T_sunburst and T_face are the time durations for which the sunburst and the face were exclusively visible. (Left) Data from monkeys and (Right) data from experimentally naive human subjects. Note that predominance is systematically related to the spatial frequency content of the sunburst pattern for both monkeys and humans; as the sunburst is blurred to greater extents, it is perceived dominant for a decreasing proportion of time.

Figure 3

Neural responses during passive viewing and during the behavioral task. (A) Response selectivity of an IT neuron. Effective stimuli were the two butterfly images, while almost all other tested images (30 tested, 4 shown) elicited little or no response from the cell. Each plot shows aligned rasters of spikes collected just before, during, and after the presentation of the image depicted below the graph. The smooth filled lines in each plot are the mean SDFs for all trials. The dotted vertical lines mark stimulus onset and stimulus removal. (B) Example observation periods taken from the behavioral task for individual cells from monkey N (Upper) and monkey R (Lower). Observation periods during behavioral testing consisted of random combinations of nonrivalrous stimuli and rivalrous periods. Dotted vertical lines mark transitions between stimulus conditions. Rivalry periods, which could occur at any time during an observation period, are shown by the filled gray background. The horizontal light and dark bars show the time periods for which the monkey reported exclusive visibility of the left-lever (sunburst) and right-lever (e.g., butterfly or monkey face) objects. Note that during rivalry the monkey reports changes in the perceived stimulus with no concomitant changes of the displayed images. Such perceptual alternations regularly followed a significant change in the neurons’ activity, as shown by the individual spikes in the middle of each plot and by the SDFs below the spikes. Note the similarity of the responses elicited by the unambiguous presentation of the effective and ineffective stimuli (white regions) with those responses elicited before either stimulus becomes perceptually salient during rivalrous stimulation (gray region).

Figure 4

Cell activity sorted by the dominant percept during nonrivalrous and rivalrous conditions. (A Upper) Averaged responses to the monocularly presented ineffective and effective stimuli. Above each graph is a pictorial representation of the visual stimuli presented. At time zero, depicted by the dotted line, the stimulus changed from either a blank screen or a mixed-object (data not shown) to the ineffective (Left) or effective (Right) stimulus. The cell fired only in response to the butterfly pattern. Presentation of the sunburst had little or no effect on the neuron’s activity. (Lower) Response of the cell just before and after the onset of rivalrous stimulation, with the effective stimulus presented to one eye and the ineffective to the other. The data are sorted based the monkey’s perceptual report: trials in which the monkey first reported seeing the ineffective stimulus (Left) and those for which the monkey first reported seeing the effective stimulus (Right). In these conditions, the stimuli presented are identical, but the recorded cell response correlates well with the monkey’s reported percept. (B) Data collected using the suppression paradigm. The nonrivalrous trials (Upper) show that this cell consistently responded to the effective stimulus and not at all to the ineffective stimulus. The flash suppression trials are similar to the rivalry trials shown in A, except that preceding the rivalrous stimulation, either the effective stimulus (Lower Left) or ineffective stimulus (Lower Right) was previously presented monocularly. Rivalry onset, marked by the dotted vertical line, thus consisted of adding either the ineffective or effective stimulus to the rivalrous pair. Following rivalry onset, the monkey’s reported percept consistently switched to the newly presented stimulus, and the previewed stimulus was perceptually suppressed. Using this paradigm, phenomenal suppression was especially effective, and cell activity during the onset of rivalrous stimulation closely mirrored that during the nonrivalrous controls.

Figure 5

(A) Examples of different response types of neurons in STS and IT. While some cells elicited relatively sustained responses to visual stimuli (e.g., cells r115 and n034), others exhibited a periodic bursting behavior (e.g., n039 and r105), or a highly transient response (e.g., r027 and r083). (B) Mean normalized cell responses in the nonrivalrous (50 cells), rivalrous (24 cells), and flash suppression (33 cells) conditions, for the effective (solid line) and ineffective (dotted line) trials. In all conditions, average cell response in the effective trials was elevated over response during the ineffective trials. (C) Scatter diagram of average cell responses for all tested neurons. For visualization purposes, only projections of the response vectors onto the two first components, C1 and C2, are presented. Each marker represents the mean of all ineffective (·) and effective (○) trials for a given cell. The distance (exemplified by the solid lines for three of the cells) between almost all pairs of responses are statistically significant (see below). (D) Separation of mean responses to the perceived effective and ineffective stimuli for the three stimulation conditions. Each individual response is represented by an eight-element vector. Separation is given by the Mahalanobis distance (12), D = , where μ₁ and μ₂ are the mean response vectors for the effective and ineffective trials, respectively, and Σ is the covariance matrix of the eight-dimensional response vectors. Because the two response types usually had different variances, Σ was replaced by its unbiased estimate, S_u = (n₁S₁ + n₂S₂)/(n − 2), where S₁ and S₂ are the covariance matrices for responses to the effective and ineffective stimulus, respectively, n₁ and n₂ are the number of presentations of each stimulus type, and n = n₁ + n₂. The significance of this variance-weighted distance was assessed by means of the Hotelling T² statistic (13) given by (n₁n₂/n)D² = (n₁n₂/n)(μ₁ − μ₂)′Σ⁻¹(μ₁ − μ₂), which relates to the F distribution by [n₁n₂(n − p − 1)/n(n − 2)p]D² ∼ F_p,n−p−1, where p stands for the dimensionality of the response space. The numbers in the top right of each plot show the proportion of cells for which the response in the ineffective and effective trials was significantly different at the α = 0.05 level. It should be noted that the high percentages of modulating neurons reported here was not due to the specific multivariate analysis. Similar results, in terms of proportions of significantly modulating cells, were obtained by the more traditional analysis of counting the number of spikes occurring in individual trials. Computing mean rates, however, requires arbitrary decisions pertaining the time window over which these rates must be computed when neurons show highly variable temporal modulations.