Seeing speech and seeing sign: insights from a fMRI study

Abstract

In a single study, silent speechreading and signed language processing were investigated using fMRI. Deaf native signers of British sign language (BSL) who were also proficient speechreaders of English were the focus of the research. Separate analyses contrasted different aspects of the data. In the first place, we found that the left superior temporal cortex, including auditory regions, was strongly activated in the brains of deaf compared with hearing participants when processing silently spoken (speechread) word lists. In the second place, we found that within the signed language, cortical activation patterns reflected the presence and type of mouth action that accompanied the manual sign. Signed items that incorporated oral as well as manual actions were distinguished from signs using only manual actions. Signs that used speechlike oral actions could be differentiated from those that did not. Thus, whether in speechreading or in sign language processing, speechlike mouth actions differentially activated regions of the superior temporal lobe that are accounted auditory association cortex in hearing people. One inference is that oral actions that are speechlike may have preferential access to 'auditory speech' parts of the left superior temporal cortex in deaf people. This could occur not only when deaf people were reading speech, but also when they were processing a signed language. For the deaf child, it is likely that observation of speech helps to construct and to constrain the parameters of spoken language acquisition. This has implications for programmes of intervention and therapy for cochlear implantation.