A Laboratory Study of the McGurk Effect in 324 Monozygotic and Dizygotic Twins

Abstract

Multisensory integration of information from the talker's voice and the talker's mouth facilitates human speech perception. A popular assay of audiovisual integration is the McGurk effect, an illusion in which incongruent visual speech information categorically changes the percept of auditory speech. There is substantial interindividual variability in susceptibility to the McGurk effect. To better understand possible sources of this variability, we examined the McGurk effect in 324 native Mandarin speakers, consisting of 73 monozygotic (MZ) and 89 dizygotic (DZ) twin pairs. When tested with 9 different McGurk stimuli, some participants never perceived the illusion and others always perceived it. Within participants, perception was similar across time (r = 0.55 at a 2-year retest in 150 participants) suggesting that McGurk susceptibility reflects a stable trait rather than short-term perceptual fluctuations. To examine the effects of shared genetics and prenatal environment, we compared McGurk susceptibility between MZ and DZ twins. Both twin types had significantly greater correlation than unrelated pairs (r = 0.28 for MZ twins and r = 0.21 for DZ twins) suggesting that the genes and environmental factors shared by twins contribute to individual differences in multisensory speech perception. Conversely, the existence of substantial differences within twin pairs (even MZ co-twins) and the overall low percentage of explained variance (5.5%) argues against a deterministic view of individual differences in multisensory integration.

Keywords: McGurk effect; audiovisual fusion; behavioral genetics; multisensory integration; twin studies.

FIGURE 1

The distribution and reliability of susceptibility to the McGurk effect. (A) For a single participant (MZ Twin 153a) the mean response across stimuli (row “Mean”) and for each individual stimulus (successive rows) are shown. For each stimulus presentation, the participant reported a percept, which was classified as auditory (AUD, red), McGurk (McG, green), visual (VIS, blue), or other (OTH, gray). The colored portions of each bar illustrate the percentage of each response type across repeated presentations of the same stimulus. For instance, the red color of the bar labeled “S1” indicates that the participant reported an auditory percept for every presentation of stimulus “S1.” (B) For each participant, the mean percentage of McGurk reports across all stimuli (“% McGurk”) was calculated across stimuli, equivalent to the width of the green bar in the first row of (A). Participants were ordered by % McGurk and plotted, one symbol per participant, orange for MZ twins, blue for DZ. (C) To assess reliability, for each participant the mean percentage of McGurk reports to even and odd trials were plotted, one symbol per participant, orange for MZ twins, blue for DZ. (D) In a follow-up experimental session approximately 2 years after the initial testing session, participants responded to the same stimuli. To assess reliability, for each participant the mean percentage of McGurk reports at the initial and follow-up sessions were plotted, one symbol per participant, orange for MZ twins, blue for DZ.

FIGURE 2

Comparing McGurk susceptibility between and across co-twins. (A) For a single participant (MZ Twin 153a) the mean response across stimuli (row “Mean”) and for each individual stimulus (successive rows) are shown. For each stimulus presentation, the participant reported a percept, which was classified as auditory (AUD, red), McGurk (McG, green), visual (VIS, blue), or other (OTH, gray). The colored portions of each bar illustrate the percentage of each response type across repeated presentations of the same stimulus. For instance, the red color of the bar labeled “S1” indicates that the participant reported an auditory percept for every presentation of stimulus “S1.” (B) For the other co-twin in this pair (MZ Twin 153b) a different pattern of responses was noted. For instance, for stimulus “S1”, a majority of participant reports corresponded to the McGurk percept. (C,D) Co-twins MZ Twin 87A and 87B showed more similar responses across stimuli. (E,F) To examine individual twin pairs, the % McGurk reports within each twin pair was calculated, and the deviation from this mean plotted by showing two circles (one for each co-twin) connected by a vertical line. The average length of the vertical lines indicates the magnitude of the intraclass correlation. The left plot shows all MZ twins (orange circles), the right plot shows all DZ twins (blue circles). Within each plot, twin pairs were ordered by mean% McGurks, from least to most. For instance, the left-most orange symbol pair shows two co-twins both with 0% McGurk, while the right-most blue symbol shows two co-twins with 97 and 100% McGurk.