Prior knowledge of illumination for 3D perception in the human brain

Abstract

In perceiving 3D shape from ambiguous shading patterns, humans use the prior knowledge that the light is located above their head and slightly to the left. Although this observation has fascinated scientists and artists for a long time, the neural basis of this "light from above left" preference for the interpretation of 3D shape remains largely unexplored. Combining behavioral and functional MRI measurements coupled with multivoxel pattern analysis, we show that activations in early visual areas predict best the light source direction irrespective of the perceived shape, but activations in higher occipitotemporal and parietal areas predict better the perceived 3D shape irrespective of the light direction. These findings demonstrate that illumination is processed earlier than the representation of 3D shape in the visual system. In contrast to previous suggestions, we propose that prior knowledge about illumination is processed in a bottom-up manner and influences the interpretation of 3D structure at higher stages of processing.

Fig. 1.

Stimuli and behavioral performance. (A) Examples of stimuli. Each image is interpreted as a convex or concave ring lit from one of four light directions. All eight image types (a–h) are ambiguous. For instance, image ”a” can be interpreted as a convex ring with a light source located above-left or as a concave ring with a light below-right. One of the elements of the ring, randomly chosen from one of six possible locations (numbered 1 through 6 in image “a”), has a shape opposite to that of the ring. In the behavioral task, observers were asked to report the perceived shape of this odd element. (B) Four classes of stimuli. To simplify the description of the stimuli in this article, we adopt the convention that the depicted shape of a stimulus is that consistent with a light coming from above. Following this convention, images “a,” “b,” “g,” and “h” will be referred to as convex rings and images “c,” “d,” “e,” and “f” as concave rings. The four main classes of stimuli are assigned different color codes: pink for convex shape lit from the left, green for concave-right, orange for concave-left, and blue for convex-right. (C) Behavioral performance in discriminating the shape of the odd element. The plot shows the probability that observers reported a convex ring (thus a concave odd element) as a function of light direction. In this plot, all six possible locations of the odd elements were pooled. The most ambiguous images were “c” and “g.” The solid line is the best fit of a scaled cosine function to illustrate the bias to above-left for the assumed light direction. Error bars are SEs across observers (n = 7).

Fig. 2.

Shape-from-shading responsive regions. Group GLM map across subjects (n = 7) representing areas that were significantly more activated for shape-from-shading than scrambled stimuli [P (Bonferroni corrected) < 0.05]. The functional activations are superimposed on flattened cortical surfaces of the left and right hemispheres. The sulci are coded in darker gray than the gyri.

Fig. 3.

Shape Classifier. MVPA for the classification of Shape (convex vs. concave) from fMRI data. (A) Classifier 1 compares activations for convex and concave stimuli when light is located 67.5° on the left: that is images “b” and “f” shown in Fig. 1A. Classifier 2 compares activations for stimuli when the light was 22.5° on the left: that is for images “a” and “e,” classifier 3 for images “h” and “d,” and classifier 4 for images “g” and “c” (Fig. 1). (B) Expected classification accuracies of the Shape Image Model. This model takes into account only the variation in pixel intensities between a pair of images. (C) Expected classification accuracies of the Shape Behavior Model. This model is based on the ability to discriminate convex and concave shapes as measured behaviorally for each observer (Fig. 1C) and predicts a nonuniform performance across the four lighting conditions. (D) Classification accuracies for the four lighting directions across ROI. The mean classification accuracy is based on 100 voxels per area. Error bars indicate SEM across observers (n = 7). The dashed line indicates the chance classification level (50%).

Fig. 4.

Light Classifier. MVPA for the classification of Light (left vs. right). (A) Classifier 1 compares activations related to left-lit convex stimuli (67.5°) vs. right-lit convex stimuli (67.5°). Classifier 2 compares activations related to left-lit convex stimuli (22.5°) vs. right-lit convex stimuli (22.5°). Classifier 3 compares activations related to right-lit concave stimuli (22.5°) vs. left-lit concave stimuli (22.5°). Finally, Classifier 4 compares activations related to right-lit concave stimuli (67.5°) vs. left-lit concave stimuli (67.5°). (B). Expected classification of the Light Image Model. This model takes only into account the variation in pixel intensities between a pair of images. (C) Expected classification accuracies of the Light Behavior Model. This model performs a Bayesian inference based on the prior assumption that light comes from above-left as measured behaviorally for each observer (Fig. 1C) and predicts an asymmetrical performance across the four lighting conditions. (D) Classification accuracies for each of the four lighting directions. Mean classification accuracy is based on 100 voxels per area. Error bars indicate SEM across observers (n = 7). The dashed line indicates the chance classification level (50%).

Fig. 5.

Correlating pattern classification and behavior-based models. (A) Correlations of classification accuracies (Light classifier on the Left and Shape classifier on the Right) and the respective behavioral models for V1 (Upper) and regions along the IPS (Lower). Classification accuracies for V1 correlate well with the Light model but not with the Shape model, whereas accuracies for IPS correlate with Shape but not with Light. (B) Summary plot of correlations between classifier accuracies and Shape and Light models for each ROI. For each cortical area, the Spearman correlation of the Light classifiers with the Light Behavior Model is plotted against the correlation of the Shape classifiers with the Shape Behavior Model. Dashed lines represent significance criteria for P = 0.05 based on a permutation analysis constrained to the image information.