Predicting the consequences of our own actions: the role of sensorimotor context estimation

Abstract

During self-generated movement it is postulated that an efference copy of the descending motor command, in conjunction with an internal model of both the motor system and environment, enables us to predict the consequences of our own actions (von Helmholtz, 1867; Sperry, 1950; von Holst, 1954; Wolpert, 1997). Such a prediction is evident in the precise anticipatory modulation of grip force seen when one hand pushes on an object gripped in the other hand (Johansson and Westling, 1984; Flanagan and Wing, 1933). Here we show that self-generation is not in itself sufficient for such a prediction. We used two robots to simulate virtual objects held in one hand and acted on by the other. Precise predictive grip force modulation of the restraining hand was highly dependent on the sensory feedback to the hand producing the load. The results show that predictive modulation requires not only that the movement is self-generated, but also that the efference copy and sensory feedback are consistent with a specific context; in this case, the manipulation of a single object. We propose a novel computational mechanism whereby the CNS uses multiple internal models, each corresponding to a different sensorimotor context, to estimate the probability that the motor system is acting within each context.

Fig. 1.

Schematic diagram of the apparatus used in each condition of Experiment 1 and in Experiment 2. Experiment 1: In all conditions subjects held a cylindrical object in their right hand. In condition 1 (Robot), the object was attached to the robot, which produced the load force on the object. In condition 2 (self-produced; right hand), subjects were required to pull down on the object, which was fixed in a clamp, to track the target load waveform. In condition 3 (self-produced; left hand), subjects were required to push the object upward from underneath with their left index finger to match the target load waveform. In condition 4 (self-produced; joystick), the object was attached to the robot and the forces produced by the robot were determined by the position of a joystick moved by the left hand. Experiment 2: An object attached to a second robotic device was held in the left hand. The motion of the left hand determined the load force on the object in the right hand. The relationship between the force acting on the left and right objects was parametrically varied between trials. See text for details.

Fig. 2.

Typical example of grip force (dashed line) and load force (solid line) traces for the four conditions of Experiment 1 taken from a single subject tracking a frequency of 3.5 Hz. The data are taken from the same 4 sec time period in each trial and have been low pass-filtered.

Fig. 3.

Average baseline (a) and amplitude gain(b) of grip force modulation against frequency for the four conditions of Experiment 1.

Fig. 4.

Average phase (a), lag (b), and coherence (c) between load force and grip force against frequency for the four conditions of Experiment 1.

Fig. 5.

Typical example of grip force (thin line) and load force (thick line) traces for four feedback gains, g, of Experiment 2 taken from a single subject tracking a frequency of 2.3 Hz. The data are taken from the same time period in each trial and have been low pass-filtered.

Fig. 6.

Baseline gain (a) and amplitude gain(b) of grip force modulation against frequency at feedback gains 1 (solid lines) and 0 (dashed lines).

Fig. 7.

Phase, lag, and coherence at different frequencies at feedback gains 1 (solid lines) and 0 (dashed lines). Average phase (a), average lag (b), average coherence (c) between load force and grip force.

Fig. 8.

Baseline gain and amplitude gain of grip force modulation at different frequencies with different force feedback coupling, g, between the robots held in each hand. Graphs show the baseline gain (solid line shows the mean) and the amplitude of grip force modulation (solid line shows linear regression fit) at different feedback gains at three different frequencies and the average over all six frequencies.

Fig. 9.

Phase, lag, and coherence at different frequencies with different force feedback coupling between the robots held in each hand. Graphs show phase, lag, and coherence at different feedback gains at three different frequencies and the average over all six frequencies. The solid line shows the quadratic fits to the data.

Fig. 10.

A model for determining the extent to which two hands are acting through a single object. For simplicity only two internal models are shown. On the left is an internal forward model that captures the relationship between the motor commands sent to the left (M_L) and right(M_R) hands and expected sensory feedback when the two hands act on a single object. On the right is shown the two internal forward models that capture the behavior when the hands are manipulating separate objects. Both models make predictions of the sensory feedback from both the left(S_L) and right(S_R) hands based on the motor commands. These predictions are then compared with the actual sensory feedback to produce the sensory prediction errors (E). The errors from each model, Ê and Ẽ, are then used to determine the probability P that each model captures the current behavior. This probability determines the extent to which the motor command to one hand can be used in predictive grip force modulation of the other hand.