Solving da Vinci stereopsis with depth-edge-selective V2 cells

Abstract

We propose a new model for da Vinci stereopsis based on a coarse-to-fine disparity energy computation in V1 and disparity-boundary-selective units in V2. Unlike previous work, our model contains only binocular cells, relies on distributed representations of disparity, and has a simple V1-to-V2 feedforward structure. We demonstrate with random-dot stereograms that the V2 stage of our model is able to determine the location and the eye-of-origin of monocularly occluded regions, and improve disparity map computation. We also examine a few related issues. First, we argue that since monocular regions are binocularly defined, they cannot generally be detected by monocular cells. Second, we show that our coarse-to-fine V1 model for conventional stereopsis explains double matching in Panum's limiting case. This provides computational support to the notion that the perceived depth of a monocular bar next to a binocular rectangle may not be da Vinci stereopsis per se [Gillam, B., Cook, M., & Blackburn, S. (2003). Monocular discs in the occlusion zones of binocular surfaces do not have quantitative depth--a comparison with Panum's limiting case. Perception 32, 1009-1019.]. Third, we demonstrate that some stimuli previously deemed invalid have simple, valid geometric interpretations. Our work suggests that studies of da Vinci stereopsis should focus on stimuli more general than the bar-and-rectangle type and that disparity-boundary-selective V2 cells may provide a simple physiological mechanism for da Vinci stereopsis.

Figure 1

Occlusion geometry and the role of monocular vs. binocular cells in solving da Vinci stereopsis. (A) Schematic diagram of a scene where a near surface occludes a background. The dotted lines indicate the extent to which the near surface occludes the far surface from each eye. (B) Images seen by the left and right eyes for the scene in (A) when fixation is at the near surface. (C) A special case of (B) when the binocular background is assumed to be featureless. For all panels, gray squares indicate binocular regions, and black and white squares represent left- and right-eye-only monocular regions, respectively. In (B) and (C), the dotted lines indicate correspondence between two eyes’ images. Ovals indicate the RFs of monocular cells, with dashed and solid lines representing left- and right-eye-only RFs, respectively. The vertical dimension of RFs is not shown to scale. Only for the special case in (C) can the relative responses of the left-and right-eye-only monocular cells determine the monocular regions.

Figure 2

Stimuli used by Nakayama and Shimojo (1990). Each stimulus is composed of a binocular rectangle and monocular bar. Nakayama and Shimojo (1990) classified the four possible configurations into two valid (A and B) and two invalid (C and D) cases. Redrawn from Nakayama and Shimojo (1990).

Figure 3

Schematic representation of the V1–V2 circuitry in our model. (A) An example V2 cell that receives inputs from two V1 cells with a preferred disparity of 2 pixels to the left and two V1 cells with a preferred disparity of 0 pixels to the right. The farther of the two preferred disparities, 2 pixels in this example, is termed the cell’s da Vinci disparity. (B) The tripartite receptive field of the model V2 cell in (A).

Figure 4

Three types of population responses of our model V2 cells. Each plot shows the responses of all V2 cells for a given image location. The peak response is below the diagonal line (A), on the diagonal line (B), or above the diagonal line (C). These three types of population responses indicate a left-eye-only monocular region, a binocular region, and a right-eye-only monocular region, respectively (see text for a detailed explanation).

Figure 5

Modeling double matching in Panum’s limiting case. (A) Psychophysical data of one subject reprinted from Gillam et al. (2003) with permission from Pion Limited, London. The perceived disparity of the monocular line is plotted as a function of the separation between the monocular and binocular lines. (B) Analytical results for the single-scale disparity energy model plotted for four scales with σ = 2, 4, 8, and 16 min. (C) Simulation results from the multi-scale, coarse-to-fine disparity energy model.

Figure 6

Valid geometric interpretations for both valid and “invalid” stimuli in Fig. 2. (A) Scene that generates the two valid cases in Fig. 2. (B) The retinal half-images of the scene in (A). (C) Scene that generates the two “invalid” cases in Fig. 2. (D) Another scene that generates the two “invalid” cases in Fig. 2. (E) The retinal half-images of the scenes in (C) and (D). Rectangles delineated by dotted lines in A, C and D represent featureless opaque surfaces. Other figure conventions are the same as in Fig. 1.

Figure 7

A valid geometric interpretation for an “invalid” random-dot stimulus in Shimojo and Nakayama (1990). The scene depicted in (A) generates the half-images in (B) used in Shimojo and Nakayama (1990). Figure conventions are the same as in Fig. 1.

Figure 8

Ocularity (eye-of-origin) maps computed at the V2 stage of our model for two random dot stereograms. (A) Simulation results from a stereogram whose middle third had a near disparity of −4 pixels and the remaining two thirds had a 0 disparity. (B) Simulation results from a stereogram whose middle third had a far disparity of 4 pixels and the remaining two thirds had a 0 disparity. In each part, the top panel shows the ideal ocularity map, with gray color representing binocular regions, and black and white colors representing left- and right-eye-only monocular regions, respectively. The bottom panel shows the raw computed ocularity map where a continuous gray scale represents the normalized difference between the two preferred disparities of the most responsive cell at each image location. Gray color represents 0 difference, and black and white colors represent negative and positive differences, respectively. A threshold can be used to convert each raw map shown here into a specific ocularity map for comparison with the corresponding ideal map.

Figure 9

Disparity maps computed at both the V1 and V2 stages of our model for the same random dot stereograms used in Fig. 8. (A) Results from the stereogram whose middle third has a near disparity of −4 pixels. (B) Results from the stereogram whose middle third has a far disparity of 4 pixels. In each part, the first and second rows show the V1 and V2 results, respectively. Plots in the first column show disparity as a function of the horizontal image position. The black traces are the ideal disparity maps; the monocular regions (marked by dotted vertical lines) take the disparity values of the background surfaces. The gray traces are the computed disparities. Plots in the second column are gray scale representation of computed disparities as a function of both horizontal and vertical image positions.

Figure 10

Effect of varying the number of V1 cells connected to a V2 cell. The percent error improvement is the difference between the V1 error (in disparity map computation) and the V2 error divided by the V1 error, averaged over 10 stereograms. Circles and squares represent results for stimuli with −4 (near) and 4 (far) pixels of disparity, respectively.

Figure 11

Simulations of disparity-boundary-selectivity in V2. (A) Recordings of two real disparity-boundary-selective V2 cells reprinted from (von der Heydt et al., 2000) with permission from Elsevier. Responses are shown as a function of the relative position between the cells’ RFs (small rectangles) and the central figures (large rectangles) of random dot stereograms. Each cell is tuned to a specific disparity edge in the stereograms. (B) Disparity-boundary-selectivity for two of our model V2 cells. (C) A half-maximum threshold is applied to (B) to eliminate the intermediate responses.

Figure 12

Simulations of two-dimensional disparity tuning of disparity-boundary-selective V2 cells. (A) Recordings of two real disparity-boundary-selective V2 cells reprinted from (von der Heydt et al., 2000) with permission from Elsevier. The RF of each cell was always aligned with a disparity boundary in a random dot stereogram (shown in the schematic to the right in the top row). Each plot shows the responses of a single V2 cell to various combinations of the foreground and background disparities. (B) Disparity tuning for two of our model V2 cells. (C) A half-maximum threshold is applied to (B) to eliminate the intermediate responses.

Figure 13

Ocularity map and disparity map computations of our model with the half-maximum threshold used in Figs. 11 and 12. The stereogram is the same as that in Figs. 8A and 9A, with a near disparity of −4 pixels for the middle third of the image. (A) Computed ocularity map at the V2 stage of the model. (B) Computed disparity maps at the V1 and V2 stages of the model. The figure conventions are the same as in Figs. 8 and 9.