How each movement changes the next: an experimental and theoretical study of fast adaptive priors in reaching

Abstract

Most voluntary actions rely on neural circuits that map sensory cues onto appropriate motor responses. One might expect that for everyday movements, like reaching, this mapping would remain stable over time, at least in the absence of error feedback. Here we describe a simple and novel psychophysical phenomenon in which recent experience shapes the statistical properties of reaching, independent of any movement errors. Specifically, when recent movements are made to targets near a particular location subsequent movements to that location become less variable, but at the cost of increased bias for reaches to other targets. This process exhibits the variance-bias tradeoff that is a hallmark of Bayesian estimation. We provide evidence that this process reflects a fast, trial-by-trial learning of the prior distribution of targets. We also show that these results may reflect an emergent property of associative learning in neural circuits. We demonstrate that adding Hebbian (associative) learning to a model network for reach planning leads to a continuous modification of network connections that biases network dynamics toward activity patterns associated with recent inputs. This learning process quantitatively captures the key results of our experimental data in human subjects, including the effect that recent experience has on the variance-bias tradeoff. This network also provides a good approximation of a normative Bayesian estimator. These observations illustrate how associative learning can incorporate recent experience into ongoing computations in a statistically principled way.

Figure 1.

The reaching task. A, Subjects reached to visual targets with virtual visual feedback of the index fingertip (black dot) available ∼100 ms after movement onset. For experiment 1, the central gray target is both the probe target and the center of the context-target distributions. For experiment 2, all seven probe targets were used and the center target was located either at 150°, as shown, or at 60° (randomized across subjects). The initial movement direction, θ_MV, was determined 100 ms after movement onset. B–D, Example trial blocks for three context conditions in experiment 1 (black circles, context trials; white circles, probe trials). Insets, Context target histograms.

Figure 2.

Experimental results. A, Reach variability of the initial movement direction in experiment 1. B, Reach bias in experiment 2, with positive values reflecting bias toward the repeated-target position. The value of zero bias at the repeated-target position (0°) is nominal, since bias is defined as the error toward that location; there was no significant change across context conditions in the average angular error at this target (F_(2,21) = 1.13, p = 0.34). C, Angular error in the CW and CCW trial blocks of experiment 3. Error bars represent SEs.

Figure 3.

Variance–bias tradeoff in the normative Bayesian model. A, The components of the normative Bayesian model. B, Change in MAP estimator variance as a function of context variance. C, Output bias of the MAP estimator as a function of input location and context variance.

Figure 4.

Evolution of reach bias across trial blocks. Data show the mean (±SE) across subjects of the bias at the ±90° probe targets for each trial block within a session, separately for each context.

Figure 5.

Comparison of adaptive Bayesian model and experimental results. A, B, Reach bias (A) and reach variance (B) as a function of probe target angle and context target distribution for experiment 2. Dashed lines show mean results from the iterative Bayesian model fit to each experimental session in experiment 2. Solid lines show the observed data (mean ± SE). C, Reach variance for experiment 1. Dashed lines show predictions from the adaptive Bayesian model with the mean per session parameters used in A and B. Behavioral data in A and C are replotted from Figure 2.

Figure 6.

Variance–bias tradeoff in the adaptive network model. A, The network model. Top, Mean input activation (black dashed line) and a single example input (gray lines) for a target at θ = 0°. Bottom, Recurrent connections reflect network topography. B, Change in network output variance as a function of context variance. C, Bias in network output as a function of input location and context variance.

Figure 7.

Comparison of network behavior (colored lines) and experimental results (gray lines) when the network is simulated with the same trials sequences that subjects experienced in experiments 1–3. A, Reach variance in experiment 1. B, Reach bias in experiment 2. C, Reach variance in experiment 2. D, Reach bias in experiment 3. E, Evolution of reach bias across trial blocks in experiment 2. Behavioral data are replotted from Figures 2 and 4. The same network parameters were used in all plots and were fitted to the data in A and C.

Figure 8.

Adaptive network model approximates Bayesian estimation. A, Matrix of initial recurrent weights before training, with units arranged topographically by preferred direction (PD). 0° represents the repeated target angle. B, Changes in the central portion of the weight matrix following 100 training trials with different context target distributions. C–E, Comparison of network output and a matched normative Bayesian model as a function of context variance during training and input gain during testing. C, Solid lines, Network output variance. Dashed lines, Predictions from a matched Bayesian model with likelihoods determined after training with the uniform context variance (gray curve) and priors determined by the network results with 60 Hz input gain (gray vertical bar); the Bayesian model necessarily matches the network model for those data points. D, Same plot as in C but using a network with network baseline firing rates lowered to 1 Hz. Priors for this network were also estimated with the 60 Hz input gain (not shown). E, Network output bias (solid lines) for θ = 10° compared with the prediction of the matched Bayesian model from D (dashed lines).