Learning optimal adaptation strategies in unpredictable motor tasks

Abstract

Picking up an empty milk carton that we believe to be full is a familiar example of adaptive control, because the adaptation process of estimating the carton's weight must proceed simultaneously with the control process of moving the carton to a desired location. Here we show that the motor system initially generates highly variable behavior in such unpredictable tasks but eventually converges to stereotyped patterns of adaptive responses predicted by a simple optimality principle. These results suggest that adaptation can become specifically tuned to identify task-specific parameters in an optimal manner.

Figure 1.

Evolution of within-trial adaptive behavior for random rotation trials. A, Mean hand trajectories for ±90° rotation trials in the first 10 batches averaged over trials and subjects (each batch consisted of 200 trials, ∼5% of which were ±90° rotation trials). The −90° rotation trials have been mirrored about the y-axis to allow averaging. Dark blue colors indicate early batches, green colors intermediate batches, red colors indicate later batches. B, The minimum distance to the target averaged for the same trials as A (error bars indicate SD over all trajectories and all subjects). This shows that subjects' performance improves over batches. C, Mean speed profiles for ±90° rotations of the same batches. In early batches, movements are comparatively slow and online adaptation is reflected in a second peak of the speed profile which is initially noisy and unstructured. D, The magnitude of the second peak increases over batches (same format as B). E, SD profiles for ±90° rotation trajectories computed for each trial batch. F, SD of the last 500 ms of movement. Over consecutive batches the variability is reduced in the second part of the movement.

Figure 2.

Evolution of motor responses to random target jumps. A, Mean trajectories for ±90° target jumps over batches of 200 trials, ∼5% of which were ±90° target jump trials. Dark blue colors indicate early batches, red colors indicate later batches. B, The bottom shows that subjects' performance did not significantly improve over trials. Error bars indicate SD over all trials and subjects. C, Mean speed profiles for ±90° target jumps of the same trial batches. A second velocity peak is present right from the start. D, The bottom shows the evolution of the magnitude of the second speed peak. E, SD for ±90° target jumps computed over the same trial batches. Over consecutive batches the variance remains constant. F, SD over the last 500 ms of movement.

Figure 3.

A, Rotation group. Relative cost of subjects' movements in response to ±90° visuomotor rotations. Over trial batches (200 trials) the cost of the adaptive strategy decreases. B, Target jump group. Relative cost of subjects' movements in response to ±90° target jumps. There is no improvement over trials. In both cases the costs have been computed by calculating the control command and the state space vectors from the experimental trajectories by assuming a quadratic cost function. The cost has been normalized to the average cost of the last five trial batches.

Figure 4.

Evolution of within-trial adaptation and control for ±90° random rotations in the second block of 2000 trials. A, Movement trajectories averaged over batches of 200 trials for the group that had experienced unexpected rotation trials already in the previous 2000 trials. Dark blue colors indicate early batches, red colors indicate later batches. This group shows no improvement. B, Speed profiles of the same trial batches. C, SD in the same trials. There is no trend over consecutive batches. D, Average movement trajectories averaged over batches of 200 trials for the group that had experienced unexpected target jump trials in the previous 2000 trials. This group shows learning. E, Speed profiles of the target jump group. F, SD in the same trials. The movement characteristics change over consecutive batches.

Figure 5.

Evolution of within-trial adaptive control for random rotations in the second block of 2000 trials. A, Minimum distance to target in ±90° rotation trials averaged over batches of 200 trials for the group that had experienced unexpected rotation trials already in the previous 2000 trials. This group shows no improvement. Error bars show SD over all trials and subjects. B, Mean magnitude of the second velocity peak over batches of 200 trials for the rotation group. C, SD in the last 500 ms of movement for ±90° rotations computed over the same trial batches for the rotation group. There is no trend over consecutive batches. D, Minimum distance to target in ±90° rotation trials averaged over batches of 200 trials for the group that had experienced unexpected target jump trials in the previous 2000 trials. This group shows a clear improvement. E, Mean magnitude of the second velocity peak over batches of 200 trials for the target jump group. F, SD in the last 500 ms of movement for ±90° rotations computed over the same trial batches for the target jump group. The SD clearly decreases over consecutive batches.

Figure 6.

Predictions of the adaptive optimal control model compared with movement data. Averaged experimental hand trajectories (left column), speed profiles (second column), angular momentum (third column), and trajectory variability (right column) for standard trials (black) and rotation trials [±30° (blue), ±50° (red), ±70° (green), ±90° (magenta)]. The second peak in the speed profile and the magnitude of the angular momentum (assuming m = 1 kg) reflect the corrective movement of the subjects. Higher rotation angles are associated with higher variability in the movement trajectories in the second part of the movement. The variability was computed over trials and subjects. The trajectories for all eight targets have been rotated to the same standard target and averaged, since model predictions were isotropic. The model consistently reproduces the characteristic features of the experimental curves.