Visual phonetic processing localized using speech and nonspeech face gestures in video and point-light displays

Abstract

The talking face affords multiple types of information. To isolate cortical sites with responsibility for integrating linguistically relevant visual speech cues, speech and nonspeech face gestures were presented in natural video and point-light displays during fMRI scanning at 3.0T. Participants with normal hearing viewed the stimuli and also viewed localizers for the fusiform face area (FFA), the lateral occipital complex (LOC), and the visual motion (V5/MT) regions of interest (ROIs). The FFA, the LOC, and V5/MT were significantly less activated for speech relative to nonspeech and control stimuli. Distinct activation of the posterior superior temporal sulcus and the adjacent middle temporal gyrus to speech, independent of media, was obtained in group analyses. Individual analyses showed that speech and nonspeech stimuli were associated with adjacent but different activations, with the speech activations more anterior. We suggest that the speech activation area is the temporal visual speech area (TVSA), and that it can be localized with the combination of stimuli used in this study.

Figure 1

Stimulus generation. (A) Video speech (VSP) and nonspeech (VnSP); (B) Point‐light speech (PLSP) and nonspeech (PLnSP); (C, left) An example of tiling the face into squares, scrambling triads of tiles, and the resultant tiled and scrambled video control (VC) for VSP or VnSP (C, right); and (D) Tiled and scrambled PLSP and PLnSP as point‐light control (PLC).

Figure 2

Whole‐brain activations projected onto the folded cortical surface, with the FDR threshold = 0.001, and cluster size = 25. A: VSP contrasted with its control, VC. B: VnSP contrasted with its control, VC. C: PLSP contrasted with its control, PLC. D: PLnSP contrasted with its control, PLC. (VSP, video speech; VnSP, video nonspeech; VC, video control; PLSP, point‐light speech; PLnSP, point‐light nonspeech; PLC, point‐light control).

Figure 3

Mean relative activations for video and point‐light in the ROIs for the left hemisphere (LH) in (A) V5/MT, (B) FFA, and (C) LOC, and for the right hemisphere (RH) in (D) V5/MT, (E) FFA, and (F) LOC. Simple contrasts with Bonferroni corrections on the main effects of condition and their levels of significance (A, B, C). Paired‐t tests on the interactions and their levels of significance (D, E, F). Activations extracted from the TVSA are shown in (G) without statistics. (Note x‐axis labels: C, control; SP, speech; nSP, nonspeech).

Figure 4

Group conjunction analyses for speech and non‐speech activations independent of media (i.e., video and point‐light). Activations in red are speech greater than the relevant control, independent of media. Activations in green are nonspeech greater than the relevant control, independent of media.

Figure 5

Individual participant conjunction analyses for speech and nonspeech left‐hemisphere activations independent of media (i.e., video and point‐light). Activations in burgundy are speech greater than the relevant control, independent of media. Activations in green are nonspeech greater than the relevant control, independent of media. Each individual's lipreading screening (ls) score is noted.