Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2011 Oct;32(10):1660-76.
doi: 10.1002/hbm.21139. Epub 2010 Sep 17.

Visual phonetic processing localized using speech and nonspeech face gestures in video and point-light displays

Lynne E Bernstein  1 , Jintao Jiang, Dimitrios Pantazis, Zhong-Lin Lu, Anand Joshi
Affiliations

Visual phonetic processing localized using speech and nonspeech face gestures in video and point-light displays

Lynne E Bernstein et al. Hum Brain Mapp. 2011 Oct.

Abstract

The talking face affords multiple types of information. To isolate cortical sites with responsibility for integrating linguistically relevant visual speech cues, speech and nonspeech face gestures were presented in natural video and point-light displays during fMRI scanning at 3.0T. Participants with normal hearing viewed the stimuli and also viewed localizers for the fusiform face area (FFA), the lateral occipital complex (LOC), and the visual motion (V5/MT) regions of interest (ROIs). The FFA, the LOC, and V5/MT were significantly less activated for speech relative to nonspeech and control stimuli. Distinct activation of the posterior superior temporal sulcus and the adjacent middle temporal gyrus to speech, independent of media, was obtained in group analyses. Individual analyses showed that speech and nonspeech stimuli were associated with adjacent but different activations, with the speech activations more anterior. We suggest that the speech activation area is the temporal visual speech area (TVSA), and that it can be localized with the combination of stimuli used in this study.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Stimulus generation. (A) Video speech (VSP) and non-speech (VnSP); (B) Point-light speech (PLSP) and non-speech (PLnSP); (C, left) An example of tiling the face into squares, scrambling triads of tiles, and the resultant tiled and scrambled video control (VC) for VSP or VnSP (C, right); and (D) Tiled and scrambled PLSP and PLnSP as point-light control (PLC).
Figure 2
Figure 2
Whole-brain activations projected onto the folded cortical surface, with the FDR threshold = .001, and cluster size = 25. (A) VSP contrasted with its control, VC. (B) VnSP contrasted with its control, VC. (C) PLSP contrasted with its control, PLC. (D) PLnSP contrasted with its control, PLC. (Note: video speech = VSP, video non-speech = VnSP, video control = VC, point-light speech = PLSP, point-light non-speech = PLnSP, point-light control = PLC.)
Figure 3
Figure 3
Mean relative activations for video and point-light in the ROIs for the left hemisphere (LH) in (A) V5/MT, (B) FFA, and (C) LOC, and for the right hemisphere (RH) in (D) V5/MT, (E) FFA, and (F) LOC. Simple contrasts with Bonferroni corrections on the main effects of condition and their levels of significance (A, B, D). Paired-t tests on the interactions and their levels of significance (C, E, F). Activations extracted from the TVSA are shown in (G) without statistics. (Note x-axis labels: C = control, SP = speech, nSP = non-speech.)
Figure 4
Figure 4
Group conjunction analyses for speech and non-speech activations independent of media (i.e., video and point-light). Activations in burgundy are speech greater than the relevant control, independent of media. Activations in green are non-speech greater than the relevant control, independent of media.
Figure 5
Figure 5
Individual participant conjunction analyses for speech and non-speech left-hemisphere activations independent of media (i.e., video and point-light). Activations in burgundy are speech greater than the relevant control, independent of media. Activations in green are non-speech greater than the relevant control, independent of media. Each individual's lipreading screening (ls) score is noted.
See this image and copyright information in PMC

References

    1. Amedi A, Malach R, Hendler T, Peled S, Zohary E. Visuo-haptic object-related activation in the ventral visual pathway. Nat Neurosci. 2001;4:324–330. - PubMed
    1. Andersson U, Lidestam B. Bottom-up driven speechreading in a speechreading expert: The case of AA (JK023) Ear Hear. 2005;26:214–224. - PubMed
    1. Auer ET, Jr, Bernstein LE. Enhanced visual speech perception in individuals with early onset hearing impairment. J Speech Lang Hear Res. 2007;50:1157–1165. - PubMed
    1. Auer ET, Jr, Bernstein LE, Sungkarat W, Singh M. Vibrotactile activation of the auditory cortices in deaf versus hearing adults. Neuroreport. 2007;18:645–648. - PMC - PubMed
    1. Beauchamp MS. See me, hear me, touch me: Multisensory integration in lateral occipital-temporal cortex. Curr Opin Neurobiol. 2005a;15:145–153. - PubMed

Publication types

MeSH terms