A Computational Mechanism for Seeing Dynamic Deformation

Abstract

Human observers perceptually discriminate the dynamic deformation of materials in the real world. However, the psychophysical and neural mechanisms responsible for the perception of dynamic deformation have not been fully elucidated. By using a deforming bar as the stimulus, we showed that the spatial frequency of deformation was a critical determinant of deformation perception. Simulating the response of direction-selective units (i.e., MT pattern motion cells) to stimuli, we found that the perception of dynamic deformation was well explained by assuming a higher-order mechanism monitoring the spatial pattern of direction responses. Our model with the higher-order mechanism also successfully explained the appearance of a visual illusion wherein a static bar apparently deforms against a tilted drifting grating. In particular, it was the lower spatial frequencies in this pattern that strongly contributed to the deformation perception. Finally, by manipulating the luminance of the static bar, we observed that the mechanism for the illusory deformation was more sensitive to luminance than contrast cues.

Keywords: MT; computational model; deformation; vision.

Figure 1.

A, A snapshot of a stimulus clip as used in experiment 1 (Extended Data Movie 1). B, Experiment 1 results. Error bars denote SEM (N = 7).

Figure 2.

A, Extracted area for simulation. The red-bound area was used to simulate the response of the direction-selective units. B, A pipeline of our model. C, Simulated spatiotemporal motion energy for the stimuli of experiment 1. In this panel, the range of each density plot is normalized between 0 and 1. Raw values were used for further analysis. D, Simulated responses of direction-selective units for the stimuli of experiment 1. In this panel, the range of each density plot is normalized between 0 and 1. Raw values were used for further analysis.

Figure 3.

A, An example of the kernel which was employed here. B, C, The vertical axis denotes the coefficient determination (r2) for the fitting of an exponential function to the proportion of trials with deformation reports as a function of NCC, and the horizontal axis denotes the spatial frequency of modulation of a kernel. The panel B is for spatiotemporal motion energy, and the panel C is for the response of direction-selective units. D, E, The psychophysical data of deformation reports (markers) are jointly plotted with the fitted values (lines) as a function of the spatial frequency of sinusoidal deformation. The panel D is for spatiotemporal motion energy, and the panel E is for the response of direction-selective units.

Figure 4.

A, Several snapshots of stimuli as used in experiment 2. B, Schematic explanations of the appearance of the deformation illusion of a bar on the basis of background drifting grating. C, Proportions of deformation reports as a function of the orientation of background drifting grating for each spatial frequency condition of the background grating.

Figure 5.

A, Extracted area for simulation. B, Simulated spatiotemporal motion energy for the stimuli of experiment 1. In this panel, the range of each density plot is normalized between 0 and 1. Raw values were used for further analysis. C, Simulated responses of direction-selective units for the stimuli of experiment 1. In this panel, the range of each density plot is normalized between 0 and 1. Raw values were used for further analysis.

Figure 6.

A, B, The vertical axis denotes the coefficient determination (r2) for the fitting of an exponential function to the proportion of trials with deformation reports as a function of NCC, and the horizontal axis denotes the spatial frequency of modulation of a kernel. The panel A is for spatiotemporal motion energy, and the panel B is for the response of direction-selective units. C, D, The psychophysical data of deformation reports (markers) are jointly plotted with the fitted values (lines) as a function of the spatial frequency of sinusoidal deformation. The panel C is for spatiotemporal motion energy, and the panel D is for the response of direction-selective units.

Figure 7.

Experiment 3 results. Proportions of deformation reports as a function of the luminance of a bar in stimuli. Error bars denote ±1 SEM for each of the orientation conditions of background drifting grating.