Brain function overlaps when people observe emblems, speech, and grasping

Abstract

A hand grasping a cup or gesturing "thumbs-up", while both manual actions, have different purposes and effects. Grasping directly affects the cup, whereas gesturing "thumbs-up" has an effect through an implied verbal (symbolic) meaning. Because grasping and emblematic gestures ("emblems") are both goal-oriented hand actions, we pursued the hypothesis that observing each should evoke similar activity in neural regions implicated in processing goal-oriented hand actions. However, because emblems express symbolic meaning, observing them should also evoke activity in regions implicated in interpreting meaning, which is most commonly expressed in language. Using fMRI to test this hypothesis, we had participants watch videos of an actor performing emblems, speaking utterances matched in meaning to the emblems, and grasping objects. Our results show that lateral temporal and inferior frontal regions respond to symbolic meaning, even when it is expressed by a single hand action. In particular, we found that left inferior frontal and right lateral temporal regions are strongly engaged when people observe either emblems or speech. In contrast, we also replicate and extend previous work that implicates parietal and premotor responses in observing goal-oriented hand actions. For hand actions, we found that bilateral parietal and premotor regions are strongly engaged when people observe either emblems or grasping. These findings thus characterize converging brain responses to shared features (e.g., symbolic or manual), despite their encoding and presentation in different stimulus modalities.

Figure 1

Conceptual diagram of emblematic gestures (emblems). Emblems share features with speech, since both express symbolic meaning, and with grasping, since both are hand actions.

Figure 2

Still frame examples from videos showing the experimental conditions. (A) Speech, spoken expressions matched in meaning to the emblems. (B) Emblem, symbolic gestures performed with the hand (shown: “it’s good”). (C) Grasping, grasping common objects with the hand (e.g., a stapler).

Figure 3

Activity against rest for each experimental condition. For each condition, the spatial extent of hemodynamic response departures from baseline (“activity”) across the cortex is depicted. Insets show the intraparietal sulcus from the superior vantage. The individual per vertex threshold was p < .001, corrected FWE p < .05.

Figure 4

Conjunction of activity. The overlap of brain activity is shown for each pair of experimental conditions: (A) The spatial extent of overlap highlighting observing symbolic meaning (emblem & speech), (B) purposeful hand actions (emblem & grasping), (C) spoken utterances and grasping (speech & grasping), and (D) all conditions. The individual per vertex threshold was p < .001, corrected FWE p < . 05.

Figure 5

Regions responsive to observing symbolic meaning. Shown is the neural activity (percent signal change) for each experimental condition in each anatomical region. Horizontal bars connect conditions that significantly differ. Error bars indicate standard error of the mean.

Figure 6

Regions responsive to observing manual actions. Shown is the neural activity (percent signal change) for each experimental condition in each anatomical region. Horizontal bars connect conditions that significantly differ. Errors bars indicate standard error of the mean.

Figure 7

Divergent neural activity in left and right supramarginal gyrus. Neural activity in the left SMG was strongest for observing speech and weakest for grasping, whereas in right SMG neural activity was strongest for observing grasping and weakest for speech. Horizontal bars connect conditions that significantly differ. Error bars indicate standard error of the mean.