Vertical binocular disparity is encoded implicitly within a model neuronal population tuned to horizontal disparity and orientation

Abstract

Primary visual cortex is often viewed as a "cyclopean retina", performing the initial encoding of binocular disparities between left and right images. Because the eyes are set apart horizontally in the head, binocular disparities are predominantly horizontal. Yet, especially in the visual periphery, a range of non-zero vertical disparities do occur and can influence perception. It has therefore been assumed that primary visual cortex must contain neurons tuned to a range of vertical disparities. Here, I show that this is not necessarily the case. Many disparity-selective neurons are most sensitive to changes in disparity orthogonal to their preferred orientation. That is, the disparity tuning surfaces, mapping their response to different two-dimensional (2D) disparities, are elongated along the cell's preferred orientation. Because of this, even if a neuron's optimal 2D disparity has zero vertical component, the neuron will still respond best to a non-zero vertical disparity when probed with a sub-optimal horizontal disparity. This property can be used to decode 2D disparity, even allowing for realistic levels of neuronal noise. Even if all V1 neurons at a particular retinotopic location are tuned to the expected vertical disparity there (for example, zero at the fovea), the brain could still decode the magnitude and sign of departures from that expected value. This provides an intriguing counter-example to the common wisdom that, in order for a neuronal population to encode a quantity, its members must be tuned to a range of values of that quantity. It demonstrates that populations of disparity-selective neurons encode much richer information than previously appreciated. It suggests a possible strategy for the brain to extract rarely-occurring stimulus values, while concentrating neuronal resources on the most commonly-occurring situations.

Figure 1. A neuronal population which explicitly encodes horizontal, but not vertical, disparity.

The shaded region represents the space of two-dimensional disparity on the retina . The purple disks represent the preferred 2D disparity of an idealized population of disparity sensors. Although these sensors form a one-dimensional population, all tuned to zero vertical disparity, they can nevertheless encode two-dimensional stimulus disparity, e.g. the stimulus disparity represented by the green dot, which has both a horizontal and a vertical component. (Cf figure 1 of Serrano-Pedraza & Read .)

Figure 2. Cells with obliquely oriented 2D disparity tuning surfaces are tuned to non-zero vertical disparities at non-optimal horizontal disparities.

A: 2D disparity tuning surface. The preferred 2D disparity is marked with a red circle: it has no vertical component. B: 1D disparity tuning curves showing neuron's response to vertical disparity, at the horizontal disparities marked with the red and purple lines in A. At the non-optimal horizontal disparity (purple curve), the neuron responds best to non-zero vertical disparities.

Figure 3. Example receptive fields in the two eyes.

The columns show the 5 different spatial frequencies, f; the receptive field envelope σ was set to 0.25/f. The two rows show 2 different phases φ: top row, even phase (φ = 0), bottom row, odd phase (φ = π/2). θ and Δx are chosen randomly in each plot from the values included in the population. Matlab code to generate this figure is Protocol S1.

Figure 4. Example image-pair.

These have horizontal disparity 2 pixels and vertical disparity 1 pixel. For clarity, these images are just 9×9 pixels; the actual images used in the simulations were 81×81 pixels. The colored dot marks corresponding pixels in the left and right images; the pink arrow shows the disparity vector. Matlab code to generate this figure is Protocol S2.

Figure 5. Disparity tuning surfaces for 15 example disparity-encoding neurons with different phase disparities and orientations.

Each panel represents the 2D disparity tuning surface for one neuron, that is, the mean spike count elicited from that neuron in response to stimuli with the two-dimensional disparity specified on the horizontal and vertical axes. Specifically, each panel shows W(θ,f,Δφ,Δx_enc;Δx_stim,Δy_stim) (Equation 5), as a function of Δx_stim and Δy_stim, for Δx_enc = 6pix, spatial frequency tuning f = 0.071cyc/pix, and the different θ and Δφ specified in the row/column labels. Each neuron's two-dimensional position disparity (Δx_pos,Δy_pos) is indicated at the top of each panel. This was set as in Equation 1, to ensure its preferred horizontal disparity is Δx_enc (here 6pix) and its preferred vertical disparity is 0. The white cross marks the pixel for which the spike count was highest. The fact that this empirical preferred disparity closely agrees with the desired value (6,0) shows that the position disparity successfully cancels out any vertical component introduced by the phase disparity. Matlab code: The mean response was obtained with Protocol S3, averaging over 500 stimuli, and the figure was generated with Protocol S4.

Figure 6. Average population response, W(θ,f,Δφ,Δx_enc;Δx_stim,Δy_stim), for different stimulus vertical disparities.

Only neurons with zero phase disparity are shown (the key features discussed in the text are the same for all phase disparities). The stimulus disparity is fixed in each panel, and the horizontal axis is the preferred horizontal disparity of the neurons (unlike Figure 5, where the neuron's preferred horizontal disparity was fixed in each panel and the horizontal axis was the horizontal disparity of the stimulus). Each panel shows the mean number of spikes which stimuli with this disparity elicit from 126 neurons, tuned to 21 different horizontal disparities Δx_enc and 6 orientations θ, plotted on the horizontal and vertical axes respectively. The 5 panels in each row show sets of 126 neurons tuned to 5 different preferred spatial frequencies. Thus together each row shows the mean response of the zero-phase-disparity sub-population, 630 neurons, averaged over 500 random stimuli with the same stimulus disparity. The stimulus horizontal disparity, Δx_stim, was set equal to −2 pixels throughout (marked with the arrow in each panel); the stimulus vertical disparity, Δy_stim, was set to a different value in each row, as indicated to the left of each row. The colorscale is the same as in Figure 5, indicated on the right. Matlab code: The mean responses were obtained with Protocol S3, and the figure was generated with Protocol S5.

Figure 7. Neuronal spike counts, R_test(θ,f,Δφ,Δx_enc), elicited by a single presentation of a single test image, with stimulus disparity (Δx_stim, Δy_stim) = (−2, +2).

As in Figure 6, only neurons with zero phase disparity are shown, Δφ = 0. The different panels each show 126 neurons tuned to different spatial frequencies f, while 21 preferred horizontal disparity tunings Δx_enc and 6 orientations θ are shown by the horizontal and vertical axes, respectively. In each panel, an arrow marks the neurons tuned to the horizontal disparity of the stimulus. The colorscale is the same in all panels. The average response of the population to all Gaussian-noise stimuli with this disparity was shown in Figure 6B (note different colorscale). This mean response differs from the single-stimulus response shown here because the latter is affected by stimulus-dependent variation, reflecting the properties of this particular image, and Poissonian noise on neuronal spiking. Matlab code: This figure was generated by Protocol S6.

Figure 8. Response of the population of disparity decoders (before rectification) to a test image with horizontal disparity Δx_test = −2pix, Δy_test = +2pix, marked with the cross.

Each pixel in the plot represents a decoding neuron, tuned to the 2D disparity (Δx_dec,Δy_dec) indicated on the horizontal and vertical axes. The pseudocolor represents the Pearson correlation coefficient between the activity in the encoding population elicited by the test image, and the stored “templates” representing the mean activity to stimuli with disparity (Δx_dec,Δy_dec). The disparity of the test image was correctly estimated from the peak activity in the decoding population. Matlab code: This figure was also generated by Protocol S6.

Figure 9. Results of estimating 2D stimulus disparity from the 1D disparity encoding population.

Each panel shows the distribution of the estimated disparity component (left column, red: horizontal disparity; right column, blue: vertical disparity). The rows show three different test disparities (Δx_test,Δy_test), as indicated by the black vertical lines in each column. In each case, 1000 images with the specified test disparity were generated, and their 2D disparity was estimated as being the value of (Δx_dec,Δy_dec) which gave the best match between the population activity R_test(θ,f,Δφ, Δx_enc) evoked by the test image, and the stored W(θ,f,Δφ,Δx_enc;Δx_dec,Δy_dec), as in Figure 8. The root-mean-squared error between the estimated disparity and the correct value is indicated at the top of each panel. Matlab code: The disparity estimates were obtained with Protocol S7, and the figure was generated with Protocol S8.

Figure 10. Disparity tuning surface for the disparity decoder tuned to Δx_stim = −6 and Δy_stim = 3, indicated by the cross in each panel.

The color of each pixel in the plot shows the mean response, stim,Δy_stim)>, averaged over 40 test stimuli with the disparity (Δx_test,Δy_test) specified by that pixel's position on the horizontal and vertical axes. A: for correlated stimuli. B: for anti-correlated stimuli. The same colorscale is used in both panels. Matlab code: The results were generated by Protocol S9 and the figure was plotted by Protocol S10.

Figure 11. Sketch of the model's physiological interpretation.

Disparity is initially encoded by a population tuned entirely to zero vertical disparity. A higher brain area extracts two-dimensional disparity from the activity of this population. The synaptic weights of the projection from the encoding to the decoding population store the mean activity of the encoding population to stimuli with different 2D disparity. For simplicity, synaptic connections onto only two, color-coded, decoding neurons are shown. The call-outs show examples of the 2D disparity tuning for the two populations (encoding: oriented, optimal vertical disparity is zero; decoding: isotropic, optimal vertical disparity may be non-zero).