Human Neuroimaging Reveals Differences in Activation and Connectivity between Real and Pantomimed Tool Use

Abstract

Because the sophistication of tool use is vastly enhanced in humans compared with other species, a rich understanding of its neural substrates requires neuroscientific experiments in humans. Although functional magnetic resonance imaging (fMRI) has enabled many studies of tool-related neural processing, surprisingly few studies have examined real tool use. Rather, because of the many constraints of fMRI, past research has typically used proxies such as pantomiming despite neuropsychological dissociations between pantomimed and real tool use. We compared univariate activation levels, multivariate activation patterns, and functional connectivity when participants used real tools (a plastic knife or fork) to act on a target object (scoring or poking a piece of putty) or pantomimed the same actions with similar movements and timing. During the Execute phase, we found higher activation for real versus pantomimed tool use in sensorimotor regions and the anterior supramarginal gyrus, and higher activation for pantomimed than real tool use in classic tool-selective areas. Although no regions showed significant differences in activation magnitude during the Plan phase, activation patterns differed between real versus pantomimed tool use and motor cortex showed differential functional connectivity. These results reflect important differences between real tool use, a closed-loop process constrained by real consequences, and pantomimed tool use, a symbolic gesture that requires conceptual knowledge of tools but with limited consequences. These results highlight the feasibility and added value of employing natural tool use tasks in functional imaging, inform neuropsychological dissociations, and advance our theoretical understanding of the neural substrates of natural tool use.SIGNIFICANCE STATEMENT The study of tool use offers unique insights into how the human brain synthesizes perceptual, cognitive, and sensorimotor functions to accomplish a goal. We suggest that the reliance on proxies, such as pantomiming, for real tool use has (1) overestimated the contribution of cognitive networks, because of the indirect, symbolic nature of pantomiming; and (2) underestimated the contribution of sensorimotor networks necessary for predicting and monitoring the consequences of real interactions between hand, tool, and the target object. These results enhance our theoretical understanding of the full range of human tool functions and inform our understanding of neuropsychological dissociations between real and pantomimed tool use.

Keywords: activity; connectivity; fMRI; pantomimed action; real action; tool use.

Figure 1.

Experimental setup. A, Participants performed two types of tasks, Real and Pantomime tool use, using two different tools, a fork or a knife. At the beginning of each trial (leftmost panel of each film strip), the participant began with the hand resting at the home position while viewing the tool and a slab of red putty on a white plate. In the Real condition, the participant grasped the fork or knife and made a poking action or a slicing action on the putty, respectively. In the Pantomime condition, the participant pretended to grasp the tool above it and then pretended to perform the action above the putty. B, Participants lay with the head and coils inclined to allow for direct viewing (without mirrors) toward a fixation LED (yellow star) above the workspace of the hand. C, An event-related average time course shows the phases of each trial and the activation from a sample area. Each trial began with a View phase (6 s) where the participant saw the scene before the Task instruction (“Real” or “Pantomime”) was provided auditorily and participants could begin anticipating the action during the Plan phase (12 s). Once a beep was heard, participants performed the instructed action in the Execute phase (4 s) with full visual feedback before the light was extinguished for an intertrial interval (ITI; 12 s). Many areas such as this showed a weak response to the visual preview of the object, followed by increased activation during plan and a robust response following action execution (with the expected hemodynamic lag of 4–6 s).

Figure 2.

Schematic diagram of multivariate pattern analysis (MVPA). For each patch of interest (POI), the β values were calculated for each vertex on the cortex-aligned surface. The β values in each condition and each run made up a multivariate pattern that was used as features for the pattern classification analysis. For Action-type decoding, data from knife and fork use were collapsed. A linear support vector machine (SVM) classifier was trained using the pattern from N−1 runs and was then used to predict the Action type of the condition in the remaining run. A leave-one-run-out cross-validation procedure was employed. For the Cross-tool Action type decoding, the training was performed with one kind of tool (knife or fork), but the test was performed on the other tool (fork or knife). Similar procedures were performed for Tool type decoding.

Figure 3.

Similarity and overlap of Real Use and Pantomime Use activation in the Execute phase. Activation is shown for lateral and medial inflated views of the two cerebral hemispheres analyzed with cortex-based alignment. A, Activation during the Execute > View phases for Real Use. B, Activation during the Execute > View phases for Pantomime Use. C, Overlap of Activation for Real Use (> Real view) and Pantomime Use (> Pantomime View). Overlap between Real and Pantomime Use appears in a spectrum between purple (relatively low significance) and lime green (relatively high significance). D, Blue-green areas show higher activation for Real Execute than Pantomime Execute (and also higher activation for Real Execute than Real View). Orange-yellow areas show higher activation for Pantomime Execute than Real Execute (and also higher activation for Pantomime Execute than Pantomime View).

Figure 4.

Activation time courses for left-hemisphere areas that were more active for Real Execute than Pantomime Execute (and also more active for Real Execute than Real View). Although the activation differences between Real and Pantomime Execute are expected based on the contrast used to define the regions, the time courses are presented to illustrate the magnitude activation in the three phases, the ramping up of the activation during the Plan phase for some regions, the similarity in activation levels between fork and knife use, and the absence of movement of movement-related artifacts (Culham, 2006; Barry et al., 2010). Time courses for regions of the right hemisphere were similar.

Figure 5.

Activation time courses for left-hemisphere areas that were more active for Pantomime Execute than Real Execute (and also more active for Pantomime Execute than Pantomime View). Time courses for regions of the right hemisphere were similar.

Figure 6.

Subcortical and cerebellar activation. A, Activation in thalamus and tegmentum (based on stereotaxic coordinates). Blue/green represents greater activation for the execution of Real Use than both the execution of Pantomime and the View phase. Event-related average profiles present time courses for these areas. No areas showed greater activation for the execution of Pantomime than Real Use. B, Activation in cerebellum. Contrasts shown represent activation greater for the execute phase of either the Real Use condition (blue/green) or the Pantomime Use (orange/yellow) with respect to the other condition as well as activation related to the View phase. Event-related average time courses are shown for areas active at p < 0.015.

Figure 7.

MVPA results. A, Decoding accuracy of Action types (real vs pantomimed) at the Plan and Execute phases. Data from knife and fork tool use were combined. B, Decoding accuracy of Action types cross tools at the Plan and Execute phases. For cross-tool Action type decoding, the train data set was from one tool (knife or fork) and test data set was from the other (fork or knife), and the results were averaged. C, Decoding accuracy of Tool types (knife vs fork). Data from real and pantomimed tool use was combined. The decoding accuracies were compared with chance level (50%). FDR correction was performed to correct for multiple comparisons. * indicates significant after FDR correction (q < 0.05). Error bars indicate ±1 SE.

Figure 8.

Results of PPI analyses with of left M1 as a seed area. A, Regions shown have significantly higher connectivity with left M1 during Real Plan than Pantomime Plan. B, Regions shown have significantly higher connectivity with left M1 during Real Execute than Pantomime Execute. C, Regions shown have significantly higher connectivity with left M1 during Pantomime Plan than Real Plan. D, Regions shown have significantly higher connectivity with left M1 during Pantomime Execute than Real Execute. Only regions that are significant after FDR correction (q < 0.05) are shown. The results were visualized with the BrainNet Viewer (Xia et al., 2013; http://www.nitrc.org/projects/bnv/).