fMRI-Guided transcranial magnetic stimulation reveals that the superior temporal sulcus is a cortical locus of the McGurk effect

Abstract

A compelling example of auditory-visual multisensory integration is the McGurk effect, in which an auditory syllable is perceived very differently depending on whether it is accompanied by a visual movie of a speaker pronouncing the same syllable or a different, incongruent syllable. Anatomical and physiological studies in human and nonhuman primates have suggested that the superior temporal sulcus (STS) is involved in auditory-visual integration for both speech and nonspeech stimuli. We hypothesized that the STS plays a critical role in the creation of the McGurk percept. Because the location of multisensory integration in the STS varies from subject to subject, the location of auditory-visual speech processing in the STS was first identified in each subject with fMRI. Then, activity in this region of the STS was disrupted with single-pulse transcranial magnetic stimulation (TMS) as subjects rated their percept of McGurk and non-McGurk stimuli. Across three experiments, TMS of the STS significantly reduced the likelihood of the McGurk percept but did not interfere with perception of non-McGurk stimuli. TMS of the STS was effective at disrupting the McGurk effect only in a narrow temporal window from 100 ms before auditory syllable onset to 100 ms after onset, and TMS of a control location did not influence perception of McGurk or control stimuli. These results demonstrate that the STS plays a critical role in the McGurk effect and auditory-visual integration of speech.

Figure 1.

fMRI-guided TMS of congruent and incongruent auditory–visual speech. A, Presentation of auditory–visual speech produced strong activation in the left superior temporal sulcus, shown in the partially inflated left hemisphere of two individual subjects. Colored regions show cortex active during both auditory and visual speech perception. Orange color highlights active cortex in the STS; green shows active cortex outside the STS. The dashed black line shows the fundus of the posterior STS. B, During presentation of the experiment 1 McGurk stimulus, subjects were likely to report the McGurk percept (white bar); TMS of the STS (red bar) but not control TMS (blue bar) significantly reduced the likelihood (*p < 0.05). Error bars show SEM across subjects. C, During presentation of the experiment 2 McGurk stimulus, subjects were likely to report the McGurk percept; TMS of the STS but not control TMS reduced the likelihood. D, The effectiveness of single-pulse TMS of the STS depended on the time at which it was delivered. TMS delivered near the onset of the auditory stimulus reduced the likelihood of the McGurk percept but TMS at other times did not.