Neural dynamics of audiovisual synchrony and asynchrony perception in 6-month-old infants

Abstract

Young infants are sensitive to multisensory temporal synchrony relations, but the neural dynamics of temporal interactions between vision and audition in infancy are not well understood. We investigated audiovisual synchrony and asynchrony perception in 6-month-old infants using event-related brain potentials (ERP). In a prior behavioral experiment (n = 45), infants were habituated to an audiovisual synchronous stimulus and tested for recovery of interest by presenting an asynchronous test stimulus in which the visual stream was delayed with respect to the auditory stream by 400 ms. Infants who behaviorally discriminated the change in temporal alignment were included in further analyses. In the EEG experiment (final sample: n = 15), synchronous and asynchronous stimuli (visual delay of 400 ms) were presented in random order. Results show latency shifts in the auditory ERP components N1 and P2 as well as the infant ERP component Nc. Latencies in the asynchronous condition were significantly longer than in the synchronous condition. After video onset but preceding the auditory onset, amplitude modulations propagating from posterior to anterior sites and related to the Pb component of infants' ERP were observed. Results suggest temporal interactions between the two modalities. Specifically, they point to the significance of anticipatory visual motion for auditory processing, and indicate young infants' predictive capacities for audiovisual temporal synchrony relations.

Keywords: ERP; asynchrony; audiovisual; infancy; multisensory perception; synchrony.

Figure 1

Experimental procedure of the habituation paradigm. During pre- and post-test, a sequence of a child movie was presented to control for fatigue effects. Habituation was reached by repeated presentation of the audiovisual synchronous stimulus (H1, H2, …). After the last habituation trial (HL), the novel asynchronous stimulus (AV400) was shown, followed again by the presentation of the familiar synchronous stimulus (F). Each experimental stimulus was presented as long as the infant looked at it or for a maximum duration of 30 s. To attract the child’s attention back to the screen, further sequences of the child movie were presented between habituation trials and between test trials. The child had to look at the screen for at least 5 s in order to continue with stimulus presentation.

Figure 2

Stimulus presentation of the EEG experiment and ERP epoch between −200 and 3400 ms. In asynchronous trials, the visual stream was delayed by 400 ms to the auditory stream. The ERPs of the synchrony (black line) and the asynchrony condition (gray line) are averaged across anterior electrodes for the purpose of illustration (n = 15). Note that the auditory onset was at the same point in time in both conditions. The three P2 components elicited during the entire epoch are labeled P2I, P2II, and P2III.

Figure 3

Results of the behavioral habituation paradigm. Infants (n = 45) looked to the asynchronous test stimulus (AV400) for longer on average than to the familiar synchronous stimulus as shown for the last habituation trial (HL) and the familiar test trial (F).

Figure 4

ERP components in the first 1000 ms after video onset. The ERPs for synchrony (black line) and asynchrony (gray line) are averaged across all anterior and all posterior electrodes for the purpose of illustration (n = 15). Solid vertical lines at 474 ms indicate the auditory onset in both experimental conditions. The visual clapping of the hands occurred at 874 ms in the asynchronous condition (indicated by dotted vertical lines).

Figure 5

ERPs elicited by synchronous (black line) and asynchronous (gray lines) stimuli in the first 1000 ms of stimulus presentation (n = 15). Note that the auditory event occurred at 474 ms in both experimental conditions (indicated by solid vertical lines).

Figure 6

Differential topographical map (asynchronous–synchronous trials) between 200 and 400 ms after stimulus onset (n = 15). Activity differences between the two stimuli emerged between 200 and 400 ms at posterior sites and between 300 and 400 ms at anterior electrodes.

Figure 7

Nc, N1, and P2 latency shifts (n = 15). Latency differences between synchronous and asynchronous stimuli did not differ significantly between the ERP components and were always significantly smaller than the AV asynchrony of 400 ms.