The Veiled Virgin illustrates visual segmentation of shape by cause

Abstract

Three-dimensional (3D) shape perception is one of the most important functions of vision. It is crucial for many tasks, from object recognition to tool use, and yet how the brain represents shape remains poorly understood. Most theories focus on purely geometrical computations (e.g., estimating depths, curvatures, symmetries). Here, however, we find that shape perception also involves sophisticated inferences that parse shapes into features with distinct causal origins. Inspired by marble sculptures such as Strazza's The Veiled Virgin (1850), which vividly depict figures swathed in cloth, we created composite shapes by wrapping unfamiliar forms in textile, so that the observable surface relief was the result of complex interactions between the underlying object and overlying fabric. Making sense of such structures requires segmenting the shape based on their causes, to distinguish whether lumps and ridges are due to the shrouded object or to the ripples and folds of the overlying cloth. Three-dimensional scans of the objects with and without the textile provided ground-truth measures of the true physical surface reliefs, against which observers' judgments could be compared. In a virtual painting task, participants indicated which surface ridges appeared to be caused by the hidden object and which were due to the drapery. In another experiment, participants indicated the perceived depth profile of both surface layers. Their responses reveal that they can robustly distinguish features belonging to the textile from those due to the underlying object. Together, these findings reveal the operation of visual shape-segmentation processes that parse shapes based on their causal origin.

Keywords: art; perception; perceptual organization; transparency; vision.

Fig. 1.

(A) Strazza’s sculpture The Veiled Virgin (approximately 1850), which elicits a vivid impression of a face “seen through” an overlying diaphanous veil (image: Wanita Bates, Presentation Archives). (B) Experiment 1: mean responses from 40 participants for one stimulus from the virtual painting task. Blue indicates contact responses; yellow indicates fabric responses (see SI Appendix, Fig. S1 for all stimuli). (C) The same data superimposed on stimulus image. (D) Profligacy and decisiveness of participants responses (see Materials and Methods for definitions). Green indicates mean responses for individual participants. Red indicates bootstrapped measurements for each participant.

Fig. 2.

(A) Cross-section of 3D scan of stimulus A1, i.e., the first draping of base shape A. (B) Renderings of the three base shapes A to C (left to right), as presented in Experiment 2. (C) Renderings of example drapings of the three base objects, as used in Experiment 2 (from left to right: A2, B4, C1). For the complete stimulus set, see SI Appendix, Fig. S2.

Fig. 3.

(A) Experiment 2: mean responses from 68 participants on stimulus C5. Blue indicates contact; yellow indicates fabric. (B) Data superimposed on the original image (see SI Appendix, Fig. S2 for all stimuli). (C) Profligacy and decisiveness of participants’ responses (see Materials and Methods for definitions). Green dots indicate responses for individual participants. Red dots indicate results of bootstrapping with random responses, statistically matched to individual participant data.

Fig. 4.

Label colors correspond to the three different underlying surfaces (cyan: A; yellow: B; green: C). (A) Correlations between the fabric maps for all stimuli revealing that participants’ responses were dominated by the differences between drapings. (B) Application of MDS to the correlation matrix reveals a disorderly arrangement in 2D, again reflecting differences in the shape of the fabric across stimuli. (C) Correlations between the contact maps for all stimuli, revealing greater similarities between stimuli that share the same base shapes. (D) Applying MDS to the correlation matrix reveals clear clustering in 2D MDS space. (E) Mean physical depth difference between textile and base shape for fabric and contact markings. Most points are above the diagonal, indicating larger depth offsets for fabric responses than for contact responses. (F) Ranking and selection of features in SVM classifier model: bars indicate magnitude of difference in mean of normalized feature values; color indicates sign (orange: fabric > contact; blue: contact > fabric). Transparency indicates features not used in SVM classifier. (G) SVM classifier predictions of causal assignment for all 154 segmented image regions. Coordinates are 2D tSNE visualization of each region in seven-dimensional (7D) feature space used for classification (orange: predicted fabric; blue: predicted contact; gray rings: incorrect predictions). (H) Predicted causal assignments of image regions for one stimulus (see SI Appendix, Fig. S2 for all stimuli) (orange: predicted fabric; blue: predicted contact).

Fig. 5.

(A) Experiment 3: stimulus C4, with green raster line indicating one cross section whose depth profile participants were asked to estimate. (B) Ground-truth depths along raster line (blue: underlying base shape; orange: overlying textile). (C) Mean responses across 12 participants, for the same raster line (blue: estimated base shape depths; orange: estimated textile depths). (D) Correlations between all ground-truth depths and participants’ responses for all nine raster lines (blue: base shape; yellow: fabric). Example stimulus is shown highlighted in yellow. For all stimuli, see SI Appendix, Fig. S3.