The Other-Race-Effect on Audiovisual Speech Integration in Infants: A NIRS Study

Abstract

Previous studies have revealed perceptual narrowing for the own-race-face in face discrimination, but this phenomenon is poorly understood in face and voice integration. We focused on infants' brain responses to the McGurk effect to examine whether the other-race effect occurs in the activation patterns. In Experiment 1, we conducted fNIRS measurements to find the presence of a mapping of the McGurk effect in Japanese 8- to 9-month-old infants and to examine the difference between the activation patterns in response to own-race-face and other-race-face stimuli. We used two race-face conditions, own-race-face (East Asian) and other-race-face (Caucasian), each of which contained audiovisual-matched and McGurk-type stimuli. While the infants (N = 34) were observing each speech stimulus for each race, we measured cerebral hemoglobin concentrations in bilateral temporal brain regions. The results showed that in the own-race-face condition, audiovisual-matched stimuli induced the activation of the left temporal region, and the McGurk stimuli induced the activation of the bilateral temporal regions. No significant activations were found in the other-race-face condition. These results mean that the McGurk effect occurred only in the own-race-face condition. In Experiment 2, we used a familiarization/novelty preference procedure to confirm that the infants (N = 28) could perceive the McGurk effect in the own-race-face condition but not that of the other-race-face. The behavioral data supported the results of the fNIRS data, implying the presence of narrowing for the own-race face in the McGurk effect. These results suggest that narrowing of the McGurk effect may be involved in the development of relatively high-order processing, such as face-to-face communication with people surrounding the infant. We discuss the hypothesis that perceptual narrowing is a modality-general, pan-sensory process.

Keywords: cross modal; fNIRS (functional near infrared spectroscopy); infant; other-race effect; the McGurk effect.

FIGURE 1

Location of the measurement channels in the current study.

FIGURE 2

An example of the order of stimulus presentation.

FIGURE 3

Time course of the changes in the oxygenated hemoglobin (Oxy-Hb) and deoxygenated hemoglobin (Deoxy-Hb) concentrations. Oxy-Hb and Deoxy-Hb concentrations were separately averaged in all groups during each condition in the left and right temporal regions. Panel (A) shows the results for the own-race-face condition, and (B) shows the results for the other-race-face condition. Solid lines represent the change in Oxy-Hb, and dotted lines represent the change in Deoxy-Hb. Blue lines and red lines represent the mean Z-score during the Match and McGurk trials, respectively. The vertical dashed lines at 0 and 9.6 s indicate the onset and offset of the test stimulus presentation, respectively.

FIGURE 4

Mean Z-scores for oxygenated hemoglobin (Oxy-Hb) and deoxygenated hemoglobin (Deoxy-Hb) in all groups for the left temporal (left) and right temporal (right) regions. Panel (A) shows the results for Oxy-Hb (in the left panel) and Deoxy-Hb (in the right panel) for the own-race-face condition and (B) shows the results for the other-race-face condition. Each bar represents the mean Z-score for Oxy-Hb (or Deoxy-Hb) averaged across 12–16 s after stimulus onset. Blue bars and red bars represent the results for the Match and McGurk conditions, respectively. The error bars represent the 95% confidence interval of the mean. Asterisks indicate the significance level of the statistical differences against baseline (0): *p < 0.05, **p < 0.01.

FIGURE 5

Mean total fixation times during the test phase in both the own-race-face condition and other-race-face condition. Dark gray bars represent the results for the novel trials (the presentation of auditory /pa/ with vegetable images) and light gray bars indicate the results for the familiarized trails (the presentation of auditory /ta/ with vegetable images). The error bars represent ±1 standard error of the mean. Asterisks indicate the significance level of the statistical differences: *p < 0.05, ** p < 0.01.