The effect of looming and receding sounds on the perceived in-depth orientation of depth-ambiguous biological motion figures

Abstract

Background: The focus in the research on biological motion perception traditionally has been restricted to the visual modality. Recent neurophysiological and behavioural evidence, however, supports the idea that actions are not represented merely visually but rather audiovisually. The goal of the present study was to test whether the perceived in-depth orientation of depth-ambiguous point-light walkers (plws) is affected by the presentation of looming or receding sounds synchronized with the footsteps.

Methodology/principal findings: In Experiment 1 orthographic frontal/back projections of plws were presented either without sound or with sounds of which the intensity level was rising (looming), falling (receding) or stationary. Despite instructions to ignore the sounds and to only report the visually perceived in-depth orientation, plws accompanied with looming sounds were more often judged to be facing the viewer whereas plws paired with receding sounds were more often judged to be facing away from the viewer. To test whether the effects observed in Experiment 1 act at a perceptual level rather than at the decisional level, in Experiment 2 observers perceptually compared orthographic plws without sound or paired with either looming or receding sounds to plws without sound but with perspective cues making them objectively either facing towards or facing away from the viewer. Judging whether either an orthographic plw or a plw with looming (receding) perspective cues is visually most looming becomes harder (easier) when the orthographic plw is paired with looming sounds.

Conclusions/significance: The present results suggest that looming and receding sounds alter the judgements of the in-depth orientation of depth-ambiguous point-light walkers. While looming sounds are demonstrated to act at a perceptual level and make plws look more looming, it remains a challenge for future research to clarify at what level in the processing hierarchy receding sounds affect how observers judge the in-depth perception of plws.

Figure 1. Proportions of FTV responses for both types of point-light figures and all sound conditions in Experiment 1.

Mean proportions (across observers) of FTV responses for the structure-only male plws (white) and the kinematics-only male plws (black) in the no-sound, stationary, receding, and looming conditions. Error bars indicate +- 1 SE from the mean. Proportions of FTV responses are higher in the looming condition and lower in the receding condition compared to the no-sound and stationary condition. The effect of sound is similar for both types of point-light figures.

Figure 2. Results of Experiment 2. Mean proportions correct and mean difference scores.

A. Mean proportions (across observers) of correct responses for each sound condition and for trials with receding plws and trials with looming plws. Compared to the no-sound condition (blue bars) in the looming sound condition (green bars) performance increased in trails with receding plws and decreased in trials with looming plws. In the receding sound condition (red bars), performance did not differ from the no-sound condition (blue bars). B. Mean of the differences in d-primes between the looming sound condition and the no-sound condition for trials with receding plws and between the no-sound condition and the looming sound condition for trials with looming plws (green bar) and mean of the differences in d-primes between the no-sound condition and the receding sound condition for trials with receding plws and between the receding sound condition and the no-sound condition for trials with looming plws (red bar). Difference scores for the looming sounds were positive indicating that looming sounds make an orthographic plw look more looming. Difference scores for receding sounds were not positive. Error bars in A and B indicate +- 1SE from the mean.