Tool use and the distalization of the end-effector

Abstract

We review recent neurophysiological data from macaques and humans suggesting that the use of tools extends the internal representation of the actor's hand, and relate it to our modeling of the visual control of grasping. We introduce the idea that, in addition to extending the body schema to incorporate the tool, tool use involves distalization of the end-effector from hand to tool. Different tools extend the body schema in different ways, with a displaced visual target and a novel, task-specific processing of haptic feedback to the hand. This distalization is critical in order to exploit the unique functional capacities engendered by complex tools.

Fig. 1

A compound of a side grasp and a power grasp for holding a screwdriver

Fig. 2

A view of macaque brain areas involved in basic parieto-frontal interactions for visually directed hand movements (from Matelli & Luppino, 2001). We will see that inferotemporal cortex and prefrontal cortex play a major role in modulating these interactions

Fig. 3

Schematic illustration of the paradigm used by Umiltà et al. (2008) to dissociate motion of the end-effector (jaws of the pliers) from the hand movements required to achieving it. To grasp the object in a precision pinch using normal pliers (a) the monkey has to close its hand in a power grasp, while with the reverse pliers (b) the monkey has to release a power grasp

Fig. 4

Novel tool and a sample of the stimulus object. Bottom a participant is shown grasping the object with the tool in the right hand during the overt grasp selection (OGS) task

Fig. 5

Increased neural activity associated with planning grasping actions with the hands or the tool relative to resting baseline. Selecting grips (PGS task) based on either hand or on the tool was consistently associated with activations within left aIPS and PMv, the putative homologs of the macaque AIP-F5 grasp circuit. In addition, activity was increased within and along the intra-parietal sulcus, as well as in bilateral superior parietal lobule (SPL) and dorsal premotor cortex (PMd), and in pre-SMA. Other structures were also activated only in some conditions, such as the right aIPS and the left middle frontal gyrus (MFG). (Jacobs et al., 2009)

Fig. 6

The complete FARS model. (adapted from the original FARS figure of Fagg & Arbib, 1998)

Fig. 7

The structure of the infant learning to grasp model (ILGM) (Oztop et al., 2004). The individual layers inside the gray box are trained based on a Joy of Grasping reinforcement signal arising from somatosensory feedback which increases with the stability of the current grasp

Fig. 8

The grasp affordance learning model (GAEM) derives its training inputs from ILGM. The successful grasping movements provide the basis for learning by GAEM: vertical learning, supervised; horizontal learning, competitive (Oztop et al., 2007)

Fig. 9

A simplified version of the augmented competitive queuing (ACQ) system as structured for the sequencing of actions in general. A crucial change here is that the mirror system can support the representation of multiple actions. Thus, during a self movement, the mirror system can represent both the intended action as well as any other action whose external appearance matches the actual performance. In ACQ, the Actor selects for execution the action that has the highest current priority, defined as a combination of executability and desirability. Desirability is the expected reinforcement for executing an action in the current internal state (which may combine homeostatic variables and cognitive subgoals). Estimates of desirability are updated by the Critic, which employs temporal difference (TD) learning. A crucial innovation here is that the Critic assesses not only the current action but also those apparent actions reported by the Mirror System in making its assessments