Discerning nonrigid 3D shapes from motion cues

Abstract

Many organisms and objects deform nonrigidly when moving, requiring perceivers to separate shape changes from object motions. Surprisingly, the abilities of observers to correctly infer nonrigid volumetric shapes from motion cues have not been measured, and structure from motion models predominantly use variants of rigidity assumptions. We show that observers are equally sensitive at discriminating cross-sections of flexing and rigid cylinders based on motion cues, when the cylinders are rotated simultaneously around the vertical and depth axes. A computational model based on motion perspective (i.e., assuming perceived depth is inversely proportional to local velocity) predicted the psychometric curves better than shape from motion factorization models using shape or trajectory basis functions. Asymmetric percepts of symmetric cylinders, arising because of asymmetric velocity profiles, provided additional evidence for the dominant role of relative velocity in shape perception. Finally, we show that inexperienced observers are generally incapable of using motion cues to detect inflation/deflation of rigid and flexing cylinders, but this handicap can be overcome with practice for both nonrigid and rigid shapes. The empirical and computational results of this study argue against the use of rigidity assumptions in extracting 3D shape from motion and for the primacy of motion deformations computed from motion shears.

Fig. 1.

Sample frames for simultaneous rotation about the vertical and depth axes for (A) a rigid cylinder, (B) a cylinder flexing in the image plane, and (C) a cylinder flexing in depth. The regular grids of dots in the figure are only for illustration purposes; in the experiments, dots were randomly placed after surface generation to remove density and texture cues to shape, and cylinders were presented behind a frame so that the curved edges were not visible to the observer.

Fig. 2.

Psychophysical and simulation results from experiment 1. (A) Fraction of trials perceived as deeper than a circle and plotted as a function of aspect ratio for the four conditions, slow rigid (SR), depth flex (NRD), plane flex (NRP), and fast rigid (FR), averaged across eight observers. (B) Observers’ points of subjective circularity for the four conditions in experiment 1 with the group mean. (C) The velocity contrast metric (VCM) computed as a function of the aspect ratios for the four types of stimuli. VCM gives a qualitative measure of the computed depth. (D) The aspect ratios of the shapes extracted by the trajectory space-based model for the four types of stimuli.

Fig. 3.

The two modeling approaches. (A) Computation of the velocity contrast metric (VCM) for a cross-sectional slice on the cylinder. (B) A complex but smooth feature trajectory modeled as a linear combination of sinusoidal trajectories.

Fig. 4.

Asymmetric percepts from symmetric cylinders. (A) Percentage perceived as asymmetric as a function of the aspect ratio for the three types of cylinders. Squares, rigid cylinders; diamonds, plane-flex cylinders; circles, depth-flex cylinders. Black solid lines and symbols represent compound rotation, and red dashed lines and hollow symbols represent simple rotation. (B) Asymmetry as predicted by the motion perspective model. (C) Asymmetry as predicted by the trajectory basis model.

Fig. 5.

Velocity profiles for the plane-flex cylinder with an aspect ratio of 0.71 under simple rotation (A) and compound rotation (B).

Fig. 6.

Results from experiment 3. (A) Percentage of correct responses for distinguishing different amounts of inflation and deflation (different colors for nine observers). Observers who did not reach the threshold for 90% deformation were not tested on less extreme deformations. Rigid, depth-flex, and plane-flex cylinders are represented using squares, circles, and diamonds, respectively. (B) Detection thresholds measured for inflation and deflation on the three types of cylinder averaged across four observers capable of doing the task.

Fig. 7.

Simulated performance of the two computational models while following inflation/deflation of the cylinders. Left (A and C) shows model predictions for a rigid cylinder undergoing 90% inflation, whereas Right (B and D) shows model predictions for a rigid cylinder undergoing 90% deflation. Upper shows predictions based on the motion perspective model, whereas Lower shows the predictions based on the trajectory basis model. Models made similar predictions for flexing cylinders.