Fingerspelling, signed language, text and picture processing in deaf native signers: the role of the mid-fusiform gyrus

Abstract

In fingerspelling, different hand configurations are used to represent the different letters of the alphabet. Signers use this method of representing written language to fill lexical gaps in a signed language. Using fMRI, we compared cortical networks supporting the perception of fingerspelled, signed, written, and pictorial stimuli in deaf native signers of British Sign Language (BSL). In order to examine the effects of linguistic knowledge, hearing participants who knew neither fingerspelling nor a signed language were also tested. All input forms activated a left fronto-temporal network, including portions of left inferior temporal and mid-fusiform gyri, in both groups. To examine the extent to which activation in this region was influenced by orthographic structure, two contrasts of orthographic and non-orthographic stimuli were made: one using static stimuli (text vs. pictures), the other using dynamic stimuli (fingerspelling vs. signed language). Greater activation in left and right inferior temporal and mid-fusiform gyri was found for pictures than text in both deaf and hearing groups. In the fingerspelling vs. signed language contrast, a significant interaction indicated locations within the left and right mid-fusiform gyri. This showed greater activation for fingerspelling than signed language in deaf but not hearing participants. These results are discussed in light of recent proposals that the mid-fusiform gyrus may act as an integration region, mediating between visual input and higher-order stimulus properties.

Figure S1

Activation for each visual input form relative to the baseline task in hearing non-signers (n = 13; voxelwise P = 0.05; clusterwise P = 0.0025). [A] FS > baseline; [B] SL > baseline; [C] TEXT > baseline; [D] PICS > baseline. On lateral renderings, activated voxels up to 15 mm beneath the cortical surface are displayed. Contiguous axial slices from TC z = − 16 to 8 mm are also shown. L = left. R = right.

Fig. 1

The British manual alphabet. Each letter of the Roman alphabet is represented by a specific hand arrangement. The British manual alphabet differs significantly from more prevalent one-handed varieties such as that used by American signers.

Fig. 2

Examples of each type of visual input form. Participants were required to make a button-press response when they saw an animal or animal name in any of the following conditions. (A) Fingerspelling: video stills illustrate each letter of the English word, ‘cat’, as it would be conveyed via fingerspelling. (B) Signed language: three video stills from the BSL sign, ‘chain’. Still images presented here are for illustration purposes only; both fingerspelled and signed stimuli were presented as color, moving video clips. (C) Text: English words were centrally presented in a black, lower-case, serif font. (D) Pictures: taken from the Snodgrass and Vanderwart (1980) normed picture set. Color versions were used in the actual experiment.

Fig. 3

Activation for each visual input form relative to the baseline task in deaf native signers (n = 13; voxelwise P = 0.05; clusterwise P = 0.0025). (A) FS > baseline; (B) SL > baseline; (C) TEXT > baseline; (D) PICS > baseline. On lateral renderings, activated voxels up to 15 mm beneath the cortical surface are displayed. Contiguous axial slices from TC z = − 16 to 8 mm are also shown. L = left. R = right.

Fig. 4

Orthographic and non-orthographic static stimuli (voxelwise P = 0.05; clusterwise P = 0.005). Main effect of Visual Input Form (TEXT vs. PICS) across deaf and hearing participants. Blue voxels represent the location of greater activation for TEXT than PICS. Orange/yellow voxels represent the location of greater activation for PICS than TEXT. Contiguous axial slices from TC z = − 24 to 40 mm are shown. L = left. R = right.

Fig. 5

Orthographic and non-orthographic visuo-dynamic stimuli (voxelwise P = 0.05; clusterwise P = 0.005). (A) Regions activated more by deaf than hearing participants across both FS and SL stimuli are shown. No regions were activated more by hearing than deaf participants. (B) Regions activated more by FS than SL stimuli across both deaf and hearing groups are shown. No regions were activated more by SL than FS. (C1) There was a significant interaction between Group and Visual Input Form in the left and right inferior temporal gyri. (C2) Graphs show percent BOLD signal change in each condition and in each group. Error bars represent standard deviation. ITG = inferior temporal gyrus. On lateral renderings, activated voxels up to 15 mm beneath the cortical surface are displayed. Contiguous axial slices from TC z = − 16 to 8 mm are also shown. L = left. R = right.