V1 neurons are tuned to perceptual borders in natural scenes

Abstract

The visual system needs to identify perceptually relevant borders to segment complex natural scenes. The primary visual cortex (V1) is thought to extract local borders, and higher visual areas are thought to identify the perceptually relevant borders between objects and the background. To test this conjecture, we used natural images that had been annotated by human observers who marked the perceptually relevant borders. We assessed the effect of perceptual relevance on V1 responses using human neuroimaging, macaque electrophysiology, and computational modeling. We report that perceptually relevant borders elicit stronger responses in the early visual cortex than irrelevant ones, even if simple features, such as contrast and the energy of oriented filters, are matched. Moreover, V1 neurons discriminate perceptually relevant borders surprisingly fast, during the early feedforward-driven activity at a latency of ~50 ms, indicating that they are tuned to the features that characterize them. We also revealed a delayed, contextual effect that enhances the V1 responses that are elicited by perceptually relevant borders at a longer latency. Our results reveal multiple mechanisms that allow V1 neurons to infer the layout of objects in natural images.

Keywords: V1; fmri; natural vision; neurophysiology; nonhuman primates.

Fig. 1.

Identifying borders in natural scenes. (A) Perceptually relevant borders as segmented by human observers. The image comes from the BSD (1). (B) Two images patches that illustrate that local contrast does not predict perceptually relevant borders. (C) Pixels with the top 10% local image contrast are shown in green (1). (D) Pixels within the top 10% energy according to a complex-cell model (yellow, Left) and top 10% energy of the VGG-19 AI model, layer “conv3_1” (Right, purple). Neither local contrast nor current V1 models capture the borders that are perceptually most relevant. (E) We computed the CRFs evoked in the visual cortex by image elements based on the local contrast or orientation energy in their (p)RF. (F) The responses of V1 neurons are enhanced when their RF falls on an elongated contour (yellow circle) in a stimulus with many line elements. This response modulation represents a recurrent influence from outside the neurons’ RF.

Fig. 2.

Object borders enhance V1 responses in humans and both awake and anesthetized monkeys. (A) Visual field coverage of pRFs in V1: To examine the coverage of the images by V1, we plotted the pRF center positions (yellow dots) across all participants. The dotted line indicates the portion of the visual field covered by at least one pRF. (B) Normalized fMRI activity elicited by the object borders (red) and other image regions (blue) in human V1 (data averaged across 45 images and four participants). See SI Appendix, Fig. S4 for the data of the individual participants. (C) Example V1 recording site. Left, RF of a representative recording site in an awake monkey (S #18 in monkey B). Right, normalized MUA responses elicited by the object borders (red) and other image regions as function of RMS contrast (blue). (D) Normalized MUA responses elicited by the object borders (red) and other image regions (blue) in area V1 as function of RMS contrast (data averaged across all recording sites in two awake monkeys). (E) Activity elicited by single neurons in V1 of anesthetized monkeys (data averaged across 53 images and two monkeys). (F and G) We binned the V1 activity in awake monkeys based on a complex cell model (F, orange) and VGG-19 model energy (G, purple). Note that the output of these models did not account for the difference in activity elicited by object borders (red) and other image regions (blue) (data averaged across four images and two awake monkeys). The equivalent result for the single units in anesthetized monkeys is shown in SI Appendix, Fig. S1C. Shaded regions denote 95% CI of the CRFs (determined by bootstrapping). Error bars indicate SEM across voxels (B) recording sites (D) or neurons (E). *** indicates P < 0.001, bootstrap test. S indicates the number of recording sites while N indicates the number of single neurons.

Fig. 3.

Perceptual borders enhance the early responses of V1 neurons. Normalized V1 response time courses in the center bin of the CRF equated for RMS contrast (A), the output of the complex cell model (B), or VGG-19 output (C). Lower panels show the difference in activity elicited by object borders and contrast/energy-matched nonborder regions. Colored arrows indicate the latency of BoM. For each model, object borders enhance MUA responses in V1 at a latency of ~50 ms. The equivalent results for single units in V1 are shown in SI Appendix, Fig. S1D.

Fig. 4.

Fast, local BoM in V1 of awake and anesthetized monkeys. (A) Examples of isolated BSD image patches matching RFs of different V1 recording sites. (B) Time course of the V1 responses of two example recording sites. Object borders elicited stronger early activity than nonborder image patches. The gray horizontal line shows the 25 to 75 ms time window of interest. (C) Distribution of early (time window, 25 to 75 ms) BoM in awake monkeys elicited by image patches across recording sites. White bar, median BoM; ***P < 0.001, Wilcoxon signed-rank test. S, number of recording sites; darker dots show the modulation of the two examples in (A). (D) Distribution of early BoM in anesthetized monkeys elicited by image patches across single neurons. N, number of single units.

Fig. 5.

Contextual BoM of the delayed V1 response. (A) To examine the role of contextual information around the V1 RF, we modified natural images ensuring that the same features were present the RFs. We either copied an image patch with an object border to a background location (Left) or removed the object from the scene by creating metamers (Middle and Right). The RF stimulus was identical in all conditions. (B) Average V1 response elicited by image regions that demarcated object borders (red; CI were too small to visualize) or were part of the background (blue). Lower panel, response difference. BoM in this experiment had a latency of 81 ms. The relative size of the patch, with respect to the median V1-RF, is shown in the Top Inset: All V1-RFs were well inside the patch. (C) The responses elicited by the metamers did not reveal significant differences.