Generalization as a behavioral window to the neural mechanisms of learning internal models

Abstract

In generating motor commands, the brain seems to rely on internal models that predict physical dynamics of the limb and the external world. How does the brain compute an internal model? Which neural structures are involved? We consider a task where a force field is applied to the hand, altering the physical dynamics of reaching. Behavioral measures suggest that as the brain adapts to the field, it maps desired sensory states of the arm into estimates of force. If this neural computation is performed via a population code, i.e., via a set of bases, then activity fields of the bases dictate a generalization function that uses errors experienced in a given state to influence performance in any other state. The patterns of generalization suggest that the bases have activity fields that are directionally tuned, but directional tuning may be bimodal. Limb positions as well as contextual cues multiplicatively modulate the gain of tuning. These properties are consistent with the activity fields of cells in the motor cortex and the cerebellum. We suggest that activity fields of cells in these motor regions dictate the way we represent internal models of limb dynamics.

Figure 1

Experimental setup and typical data. (A) Subjects hold the handle of a robot and reach to a target. The plot shows hand trajectory (dots are 10 ms apart) for typical movements to 8 targets in the null field, i.e., robot motors disengaged. (B) Typical force fields produced by the robot. Forces are plotted as a function of hand velocity. (C) Average hand trajectories (+/- SD) for movements during the initial trials in the saddle force field. (D) Simulation results for reaches in the saddle force field. (E) Hand trajectories during catch trials in the saddle field. (F) Simulation results for catch trials in saddle field. Redrawn from Shadmehr and Brashers-Krug (1997d).

Figure 2

Generalization from one configuration of the arm to another. (A) Subjects trained with the hand at the “left” workspace in the force field shown and were then tested at a “right” workspace in a different field. (B) Directional tuning for the biceps muscle. For each direction of movement, muscle activity was averaged and is plotted during the null field (solid line) and after adaptation to the force field (dashed line). The gray wedge indicates preferred direction of the muscle. The field at left produces 38° clockwise rotation in the PD of biceps. At right, PD of biceps in the null field rotated 90° with respect to null at left. However, the field at right also produced a clockwise rotation of biceps PD. (C) Performance measure (perpendicular displacement of a reach with respect to a straight line) is plotted for bin size of 16 movements. Training at left generalizes to the field at right. Redrawn from Shadmehr and Moussavi (2000).

Figure 3

Activation of bases that encode limb position and velocity as a gain field. The figure is a polar plot of activation pattern for a typical basis function in the model. The polar plot at the center represents activation for an eight-direction center-out reaching task (targets at 10 cm). Starting point of each movement is the center of the polar plot. The shaded circle represents the activation during a center-hold period and the polygon represents average activation during the movement period. The eight polar plots on the periphery represent activation for eight different starting positions. Each starting position corresponds to the location of the center of each polar plot. The preferred positional gradient of this particular basis function has a rightward direction. The preferred velocity is an elbow flexion at 62°/s. Redrawn from Hwang et al. (2003).

Figure 4

Consequences of learning internal models with bases that encode static limb position and movement direction as a gain field. Generalization patterns that are produced by these bases predict that certain tasks will be very difficult to learn. (A) Because the bases encode state of the limb in intrinsic coordinates (e.g., joint position and velocity), and associate this state to joint torques, a field that is translation invariant in Cartesian coordinates will be hard to learn. (B) Because the bases linearly encode static position of the limb, it is difficult to learn to associate movements that are in the same direction with different forces. However, when the movements are sufficiently far apart (about 14 cm), the task becomes learnable. When the center movement is in a null field, the linear encoding of static limb positions makes it so that the generalization pattern from left and right movements cancel at center, making this an easy task to learn. When the null field is placed to the right, the middle movement in field F2 generalizes to the right movement, making this task hard to learn. (C) A field where forces depend on hand position. Reaching targets are drawn as small circles. In the easy task, movements that are in the same direction (for example, from bottom target to the center target) have the same force pattern. In the hard task, movements that are in the same direction have opposite forces. Part B redrawn from Hwang et al. (2003).

Figure 5

Estimation of a generalization function from trial-by-trial patterns of error. (A) Top row: Black lines are movement errors during 192 movements (out-and-back pattern) in a standard curl field paradigm to 8 directions of targets. Sharp negative spikes are catch trials. Black lines are measured data and gray lines are fit to Eq. (4). Subjects performed 3×192 movements (3 target sets), but data for only one set is shown. Second row: In this experiment, subjects practiced in a target set that was not out-and back, but random directions. The shape of the generalization function and compliance are similar to that obtained in the first row. Third row: In this experiment, subjects trained in a force field that randomly changed from movement to movement. Despite no obvious learning trends, the generalization function is similar to other “learnable” tasks. (B) The estimated generalization function (b in Eq. 4). The generalization function implies that ∼18% of the error that was recorded for a movement toward any given direction updated the internal model for that same direction. About 12% of error was generalized to neighboring directions at 135° and 180°. The same subjects were again tested on the same field a second and a third time (2^nd and 3^rd target sets, each set 192 movements). The generalization functions for all three sets of targets are shown in (B). Little change is seen in these repeated measures. (C) The estimated compliance matrix D for each target set. Compliance matrix is plotted by multiplying D by a unit force vector that goes about a circle. The estimates change little with repeated measures. The orientation of the ellipse is consistent with previous estimates of arm stiffness (Mussa-Ivaldi et al., 1985). (D) A basis function consistent with the generalization functions. This particular basis has a preferred velocity at [0.21, 0.21] m/s, corresponding to the peak velocity for a 10 cm movement toward 45°. Dark regions indicate higher activation. Redrawn from Donchin et al. (2003).