Neural correlates of auditory-visual stimulus onset asynchrony detection

Abstract

Intersensory temporal synchrony is an ubiquitous sensory attribute that has proven to be critical for binding multisensory inputs, sometimes erroneously leading to dramatic perceptual illusions. However, little is known about how the brain detects temporal synchrony between multimodal sensory inputs. We used positron emission tomography to demonstrate that detecting auditory-visual stimulus onset asynchrony activates a large-scale neural network of insular, posterior parietal, prefrontal, and cerebellar areas with the highest and task-specific activity localized to the right insula. Interregional covariance analysis further showed significant task-related functional interactions between the insula, the posterior thalamus, and superior colliculus. Based on these results and the available electrophysiological and anatomical connectivity data in animals, we propose that the insula, via its known short-latency connections with the tectal system, mediates temporally defined auditory-visual interaction at an early stage of cortical processing permitting phenomena such as the ventriloquist and the McGurk illusions.

Fig. 1.

Subjects' performance while detecting auditory–visual synchrony–asynchrony at different intermodal delays. Means (± SEM) of percentage of correct responses (a) and response times (b) of 12 subjects. AV (open circles), Sound leading; VA (solid circles), light leading. Percentage of correct responses significantly increased (F(4,11) = 45.4;p < 0.0001 and 61.7; p < 0.0001) while response time decreased (F(4,11) = 9.9; p < 0.0034 and 18.7; p < 0.0001), as a function of intermodal delay for AV and VA, respectively. At delays of 100, 150, and 200 msec, subjects were faster and more accurate in AV than VA conditions (t test,p < 0.006).

Fig. 2.

Brain regions activated in common to both auditory–visual and visual–auditory synchrony–asynchrony detection conditions. Statistical parametric maps were superimposed on axial views of a normalized representative subject's brain.a, Right inferior parietal lobule (46, −54, 48;z = 5.15; p < 0.006);b, right ventrolateral prefrontal cortex (48, 34, 18;z = 5.60; p < 0.001);c, right anterior insular cortex (36, 24, −4;z = 6.57; p < 0.0001); andd, left cerebellar hemisphere (−28, −58, −48;z = 5.43; p < 0.002).p values corrected for multiple comparisons.

Fig. 3.

Statistical parametric map (thresholded atz > 4.63; p < 0.05 corrected for multiple comparisons) showing voxels with significant incremental rCBF response to increasing task demand superimposed on sagittal (a) and axial (b) views of a normalized representative subject's brain. Voxel with highest covariance: x = 38, y = 24, z = −4; z = 6.42; p < 0.0001.

Fig. 4.

Brain regions with significant functional interactions with the right insula during synchrony–asynchrony detection. Correlations are displayed as statistical maps (z > 3.09) superimposed on sagittal views of a normalized representative subject's brain. Coordinates (x, y, z) of voxels of maximal positive rCBF correlation with the reference voxel (36, 24, −4): a, left insula (−36, 12, 0); b, posterior midbrain (in the region of the superior colliculus; −2, −28, −12); c, right precuneus (12, −80, 48); d, right posterior thalamus (18, −22, 8); and e, right prefrontal cortex (32, 48, 20).