Structural learning in feedforward and feedback control

Abstract

For smooth and efficient motor control, the brain needs to make fast corrections during the movement to resist possible perturbations. It also needs to adapt subsequent movements to improve future performance. It is important that both feedback corrections and feedforward adaptation need to be made based on noisy and often ambiguous sensory data. Therefore, the initial response of the motor system, both for online corrections and adaptive responses, is guided by prior assumptions about the likely structure of perturbations. In the context of correcting and adapting movements perturbed by a force field, we asked whether these priors are hard wired or whether they can be modified through repeated exposure to differently shaped force fields. We found that both feedback corrections to unexpected perturbations and feedforward adaptation to a new force field changed, such that they were appropriate to counteract the type of force field that participants had experienced previously. We then investigated whether these changes were driven by a common mechanism or by two separate mechanisms. Participants experienced force fields that were either temporally consistent, causing sustained adaptation, or temporally inconsistent, causing little overall adaptation. We found that the consistent force fields modified both feedback and feedforward responses. In contrast, the inconsistent force field modified the temporal shape of feedback corrections but not of the feedforward adaptive response. These results indicate that responses to force perturbations can be modified in a structural manner and that these modifications are at least partly dissociable for feedback and feedforward control.

Fig. 1.

The effect of structural learning on the prior assumptions of the temporal shape of force fields. Possible perturbing forces are shown as points in coefficient space, with the x-axis indicating the strength and direction of the position-dependent component, whereas the y-axis indicates the strength and direction of the velocity-dependent component. The initial adaptation to any force field is biased toward the diagonal in the position/velocity coefficient space (solid arrow) (Sing et al. 2009). This bias indicates that the motor system relies on a prior probability distribution (blue cloud), which indicates that force fields with position and velocity components in the same direction are more likely than perturbations with components in opposite directions. We hypothesize that after repeated exposure to a position-dependent force field (independent of the direction of this force field), the response should now be biased toward the position-dependent axis. This indicates that prior assumption of the motor system has changed; i.e., structural learning has occurred.

Fig. 2.

Methods for experiment 1. A: 3 different kinds of force fields (FF). In all cases, the force is orthogonal to the actual movement direction. The combination force field is a mixture of both position- and velocity-dependent force fields. The position-dependent force field increases monotonically with the y-position of the hand. The velocity-dependent force field is proportional to the y-velocity of the arm. B: the experiment consisted of a pretest phase with 6 short blocks of adaptation (2 blocks shown) to the combination force field (blue). The direction of the force alternated on each block from rightward (+) to leftward (−). Force channel trials (gray) were used to monitor adaptation. This was followed by an exposure phase, with 6 blocks of adaptation to pure velocity- or position-dependent force fields (orange), alternating left/right across blocks. Finally, in the post-test phase, subjects adapted in short blocks to the combination or pure force field, which alternated in type across blocks. These blocks were counterbalanced for direction, such that sometimes, a rightward block was followed by a leftward block and sometimes by a rightward block and vice versa.

Fig. 3.

Change in the adaptation to a pure position- and velocity-dependent force field in experiments 1 and 2. Each point indicates the shape of the force response in a channel trial plotted in coefficient space. The evolution of learning after 0, 2, 6, and 10 trials of the exposure to a force field is shown. As the experiment progressed, the initial response became more position dependent and less velocity dependent. This demonstrates the effect of structural learning, such that once they experience a position-dependent force field for the first time, subsequent adaptation to such a force field was faster and more direct. No significant effect was observed for the velocity-dependent force field.

Fig. 4.

Results of experiment 1: adaptation to the combination force field before and after exposure to either a position- or a velocity-dependent force field. The channel response was measured before adaptation and at 3 time points during adaptation. A: the force that participants exert in the 2nd, 3rd, and 4th channel trials is shown for the pretest (green), after exposure to a position-dependent force field (blue), and after exposure to a velocity-dependent force field (red). After exposure to a position-dependent force field, participants exert lower forces at peak velocity and relatively higher forces in the end of the movement. After exposure to a velocity-dependent force field, the forces at peak velocity increase, but the forces in the end of the movement decreased. B: the same results plotted in coefficient space. We regressed the force traces against the position and velocity of that trial and plotted the regression coefficients for position (x-axis) against the regression coefficients for velocity (y-axis). Results are flipped and averaged across leftward- and rightward-directed force fields. The 95% confidence ellipses for the mean across participants are shown around each point. C: the velocity trajectories in the direction of movement during channel trials for the 3 conditions are shown, averaged over participants, and the shading indicates the SE across participants. D: the difference between the pre-exposure channel force and the postexposure channel force for the 2 exposure conditions is shown here. The differences have been averaged over channel trials 2–4 and clearly show the structure-specific change in the force exerted in the channel.

Fig. 5.

Results from experiment 2. A: we measured channel responses, caused by feedback mechanisms reacting to the force channel, which is at an angle to the intended movement direction. This was achieved by letting people reach to a target that was displaced laterally from straight ahead. On channel trials, the target still appeared at an eccentric angle, while the hand was constrained to move straight ahead in a force channel. During these channel trials, the cursor was rotated to move directly to the target. B: lateral force exerted into the channel for the 7° and 14° channel trials showed stronger position components after exposure to a position-dependent force field (blue trace) and stronger velocity components after exposure to a velocity-dependent force field (red). The force profile in the pretest phase (green trace) had the same stereotypical position/velocity profile. C: regression coefficients from the same time series show the same changes, with the lines shifting toward the axis of the force field experienced in the exposure phase. The ellipses indicate the 95% confidence interval for the between-participants mean. D: the differences between the pre-exposure channel force and the postexposure channel force for the 2 exposure conditions are shown here. These have been averaged over the 7° and 14° channels, and once more, the structure-specific change in the channel force can be seen clearly.

Fig. 6.

Experiment 3 shows dissociation of structural learning in feedforward and feedback control, depending on the temporal consistency of the exposure phase. A: the exposure blocks consisted of trials with a position-dependent force field (orange), whose direction (+ or −) was consistent over 12 trials or inconsistent, resulting in a negative lag-1 autocorrelation. Channel trials (gray) were interspersed randomly. Participants were tested both on adaptation to a combination force field (feedforward control) and reaction to titled channels (feedback control). B: average force trace exerted in the channel for the pretest (green), post-test after inconsistent exposure (light blue), and post-test after consistent exposure (dark blue) in the feedforward and feedback conditions. C: the regression coefficients, presented as in Figs. 4B and 5C, indicated a change in response toward a position-dependent force field. D: the change in the angle in coefficient space from pretest to post-test indicates structural learning. A negative number indicates a change of the angle toward the position axis. For feedforward adaptation, a modulation toward the position-dependent force field was only found for the consistent force field. For feedback control, modulation is found for both cases. There was a significant interaction of consistency and block type (*P < 0.05); n.s., not significant.