Perceptual integration for qualitatively different 3-D cues in the human brain

Abstract

The visual system's flexibility in estimating depth is remarkable: We readily perceive 3-D structure under diverse conditions from the seemingly random dots of a "magic eye" stereogram to the aesthetically beautiful, but obviously flat, canvasses of the Old Masters. Yet, 3-D perception is often enhanced when different cues specify the same depth. This perceptual process is understood as Bayesian inference that improves sensory estimates. Despite considerable behavioral support for this theory, insights into the cortical circuits involved are limited. Moreover, extant work tested quantitatively similar cues, reducing some of the challenges associated with integrating computationally and qualitatively different signals. Here we address this challenge by measuring fMRI responses to depth structures defined by shading, binocular disparity, and their combination. We quantified information about depth configurations (convex "bumps" vs. concave "dimples") in different visual cortical areas using pattern classification analysis. We found that fMRI responses in dorsal visual area V3B/KO were more discriminable when disparity and shading concurrently signaled depth, in line with the predictions of cue integration. Importantly, by relating fMRI and psychophysical tests of integration, we observed a close association between depth judgments and activity in this area. Finally, using a cross-cue transfer test, we found that fMRI responses evoked by one cue afford classification of responses evoked by the other. This reveals a generalized depth representation in dorsal visual cortex that combines qualitatively different information in line with 3-D perception.

Figure 1

Stimulus illustration and experimental procedures. (a) Left side: Cartoon of the disparity and/or shading defined depth structure. One of the two configurations is presented: bumps to the left, dimples to the right. Right side: stimulus examples rendered as red-cyan anaglyphs. (b) Illustration of the psychophysical testing procedure. (c) Illustration of the fMRI block design. (d) Illustration of the vernier task performed by participants during the fMRI experiment. Participants compared the horizontal position of a vertical line flashed (250 ms) to one eye against the upper vertical nonius element of the crosshair presented to the other eye.

Figure 2

Psychophysical results. (a) Behavioral tests of integration. Bar graphs represent the between-subjects mean slope of the psychometric function. * indicates p<.05. (b) Psychophysical results as an integration index. Distribution plots show bootstrapped values: the center of the ‘bowtie’ represents the median, the colored area depicts 68% confidence values, and the upper and lower error bars 95% confidence intervals.

Figure 3

Representative flat maps from one participant showing the left and right regions of interest. The sulci are depicted in darker gray than the gyri. Shown on the maps are retinotopic areas, V3B/KO, the human motion complex (hMT+/V5), and lateral occipital (LO) area. The activation on the maps shows the results of a searchlight classifier analysis that moved iteratively throughout the measured cortical volume, discriminating between stimulus configurations. The color code represents the t-value of the classification accuracies obtained. This procedure confirmed that we had not missed any important areas outside those localized independently.

Figure 4

Performance in predicting the convex vs. concave configuration of the stimuli based on the fMRI data measured in different regions on interest. The bar graphs show the results from the ‘single cue’ experimental conditions, the ‘disparity + shading’ condition, the quadratic summation prediction (horizontal red line). Error bars indicate SEM.

Figure 5

Prediction performance for fMRI data separated into the two groups based on the psychophysical results (‘good’ vs. ‘poor’ integrators). The bar graphs show the results from the ‘single cue’ experimental conditions, the ‘disparity + shading’ condition, the quadratic summation prediction (horizontal red line). Error bars indicate SEM.

Figure 6

(a) fMRI based prediction performance as an integration index for the two groups of participants in area V3B/KO. A value of zero indicates the minimum bound for fusion as predicted by quadratic summation. The index is calculated for the ‘Disparity + shading’ and ‘Disparity + binary shading’ conditions. Data are presented as notched distribution plots. The center of the ‘bowtie’ represents the median, the colored area depicts 68% confidence values, and the upper and lower error bars 95% confidence intervals. (b) Correlation between behavioral and fMRI integration indices in area V3B/KO. Psychophysics and fMRI integration indices are plotted for each participant for disparity + shading and disparity + binary luminance conditions. The Pearson correlation coefficient (R) and p-value are shown. (c) The transfer index values for V3B/KO for the good and poor integrator groups. Using this index, a value of 1 indicates equivalent prediction accuracies when training and testing on the same cue vs. training and testing on different cues. Distribution plots show the median, 68% and 95% confidence intervals. Dotted horizontal lines depict a bootstrapped chance baseline based on the upper 95^th centile for transfer analysis obtained with randomly permuted data.