Symbolic gestures and spoken language are processed by a common neural system

Abstract

Symbolic gestures, such as pantomimes that signify actions (e.g., threading a needle) or emblems that facilitate social transactions (e.g., finger to lips indicating "be quiet"), play an important role in human communication. They are autonomous, can fully take the place of words, and function as complete utterances in their own right. The relationship between these gestures and spoken language remains unclear. We used functional MRI to investigate whether these two forms of communication are processed by the same system in the human brain. Responses to symbolic gestures, to their spoken glosses (expressing the gestures' meaning in English), and to visually and acoustically matched control stimuli were compared in a randomized block design. General Linear Models (GLM) contrasts identified shared and unique activations and functional connectivity analyses delineated regional interactions associated with each condition. Results support a model in which bilateral modality-specific areas in superior and inferior temporal cortices extract salient features from vocal-auditory and gestural-visual stimuli respectively. However, both classes of stimuli activate a common, left-lateralized network of inferior frontal and posterior temporal regions in which symbolic gestures and spoken words may be mapped onto common, corresponding conceptual representations. We suggest that these anterior and posterior perisylvian areas, identified since the mid-19th century as the core of the brain's language system, are not in fact committed to language processing, but may function as a modality-independent semiotic system that plays a broader role in human communication, linking meaning with symbols whether these are words, gestures, images, sounds, or objects.

Fig. 1.

Examples of pantomimes (top two rows. English glosses–A. juggle balls; B. unscrew jar) and emblems (bottom two rows. English glosses–C. I've got it!; D. settle down).

Fig. 2.

Common areas of activation for processing symbolic gestures and spoken language minus their respective baselines, identified using a random effects conjunction analysis. The resultant t map is rendered on a single subject T1 image: 3D surface rendering above, axial slices with associated z axis coordinates, below.

Fig. 3.

Condition-specific activations for symbolic gesture (Top) and for speech (Bottom) were estimated by contrasting symbolic gestures and spoken language (minus their respective baselines) using random effects, paired two-sample t tests. Maps were rendered as indicated as in Fig. 2.

Fig. 4.

Functional connections between left hemisphere seed regions (IFG; MTG) and other brain areas. (A) Correlations that exceeded threshold for both symbolic gesture and spoken language and did not differ significantly between these conditions. Unique correlations, illustrated in (B) for speech and (C) for gesture, exceeded threshold in only one of these conditions. Lines connote z-transformed correlation coefficients >3 that met criteria. (STG, superior temporal gyrus; FUS, inferior temporal and fusiform gyri; PFC, dorsolateral prefrontal cortex; PHP, parahippocampal gyrus; CBH, cerebellar hemisphere).