Effects of self-motion on auditory scene analysis

Abstract

Auditory scene analysis requires the listener to parse the incoming flow of acoustic information into perceptual "streams," such as sentences from a single talker in the midst of background noise. Behavioral and neural data show that the formation of streams is not instantaneous; rather, streaming builds up over time and can be reset by sudden changes in the acoustics of the scene. Here, we investigated the effect of changes induced by voluntary head motion on streaming. We used a telepresence robot in a virtual reality setup to disentangle all potential consequences of head motion: changes in acoustic cues at the ears, changes in apparent source location, and changes in motor or attentional processes. The results showed that self-motion influenced streaming in at least two ways. Right after the onset of movement, self-motion always induced some resetting of perceptual organization to one stream, even when the acoustic scene itself had not changed. Then, after the motion, the prevalent organization was rapidly biased by the binaural cues discovered through motion. Auditory scene analysis thus appears to be a dynamic process that is affected by the active sensing of the environment.

Fig. 1.

Illustration of the experimental setup and trial types in experiment 1. (A) Auditory stimuli were presented to the Telehead robotic system. A loudspeaker was positioned in front of the robotic dummy head. Sounds were collected by microphones placed in the dummy head and transmitted in real-time to the listener via headphones. The head motion of the listener was tracked and could be mimicked with minimal latency by the robotic head. (B) Relative head and source positions at the start and end of each trial type. Details are provided in the main text.

Fig. 2.

Results for experiment 1. (A) Probability of two-stream responses averaged across listeners (N = 10) for the four trial types. (B) Same for Control trials, with the Self data replotted from A. Shaded areas indicate 95% confidence intervals. (C) Normalized data were computed by selecting the trials in which perception was two streams at the 10-s point. Resetting was evaluated over a 6-s time window (shaded area). (D) Estimated contributions to the resetting of (i) changes in acoustic cues at the ears, ΔA; (ii) apparent sound localization in allocentric coordinates, ΔS; or (iii) nonauditory factors related to head motion, ΔH. Those contributions were estimated for each listener by means of a linear additive model considering all trial types.

Fig. 3.

Setup and results for experiment 2a. (A) Trial types differed according to the spatial location of sound sources. Because of the spatial arrangement of loudspeakers, for Self trials, dynamic localization cues during head motion were fully correlated (identical) for A and B noises, partially correlated for Front-front trials, and anticorrelated for Front-back trials. There were also three types of No-change trials, with spatial configurations corresponding to Self, Front-front, and Front-back trials. (B) Probability of two-stream response averaged across listeners (N = 8) for the four trial types (Upper) and the corresponding normalized data (Lower).

Fig. 4.

Setup and results for experiment 2b. (A) Self and Front-back trials were as in experiment 2a, except that listeners started the trials by facing toward the side and subsequently moved toward the midline. The dynamic localization cues were as in experiment 2a, but the static binaural cues after head motion were now similar between trial types. (B) Probability of two-stream response averaged across listeners (N = 4) for the three trial types (Left) and the corresponding normalized data (Right).