The Role of Variability in Motor Learning

Abstract

Trial-to-trial variability in the execution of movements and motor skills is ubiquitous and widely considered to be the unwanted consequence of a noisy nervous system. However, recent studies have suggested that motor variability may also be a feature of how sensorimotor systems operate and learn. This view, rooted in reinforcement learning theory, equates motor variability with purposeful exploration of motor space that, when coupled with reinforcement, can drive motor learning. Here we review studies that explore the relationship between motor variability and motor learning in both humans and animal models. We discuss neural circuit mechanisms that underlie the generation and regulation of motor variability and consider the implications that this work has for our understanding of motor learning.

Keywords: motor adaptation; motor control; reinforcement learning; songbird.

Figure 1. Illustrating how variability can be conducive for motor learning

A. Each task is associated with a reward landscape in action space. B. When the reward landscape is not known, trial-and-error reinforcement learning offers a powerful strategy for finding appropriate solutions. This requires initial exploration of action space coupled with a process that reinforces rewarded actions. Initially variability is high, but as reinforcement learning proceeds (‘early ’ to ‘late ) ’ , variability is reduced as the motor system hones in on action variants associated with high reward.

Figure 2. Sources of motor variability

Motor variability can arise at all levels of the motor systems. Here we distinguish variability in central planning and control circuits, referred to as ‘planning noise’, from variability in the motor periphery, referred to as ‘execution noise’. Variability conducive for learning is more likely to originate in central circuits, which receive performance related feedback, as opposed to peripheral circuits where variability may be more difficult to reinforce and reproduce (see text).

Figure 3. Vocal variability in songbirds is generated by a basal ganglia-like circuit

Research on the courtship song of zebra finches has informed the link between motor variability and learning. A. The song is generated by the vocal control pathway comprising HVC, RA, and brainstem motor regions (red pathway). The Anterior Forebrain Pathway (blue pathway), a basal ganglia-thalamo-cortical circuit, is important for song learning, but not for producing learned song. B. Spectrograms of zebra finch song at different stages of song learning, shows that learning is associated with a gradual decrease in song variability and an increase in song quality, as defined by the similarity to the song model being imitated (not shown). Grey lines denote the song motif of the bird, which crystallizes to the same syllable sequence late in learning. C. Inactivating LMAN (left) causes a dramatic reduction in song variability in juvenile birds (right). Song spectrograms from the same bird before and immediately after LMAN inactivation. Data from (Ölveczky et al., 2005). D. Inactivating LMAN reduces the rendition-to-rendition variability of RA neurons. (left) Activity patterns of an LMAN neuron in a juvenile bird, aligned to a recognizable song motif (i.e. syllable sequence). Each row of spikes represents the activity during one rendition of the song motif. Note the high degree of rendition-to-rendition variability. (right) Recording from the same RA neuron in a juvenile singing bird with and without pharmacological inactivation of LMAN, shows a dramatic reduction in rendition-to-rendition variability in the RA neurons with LMAN silencing. Data from (Ölveczky et al., 2011).

Figure 4

Structure of motor variability predicts learning rates in a reinforcement-based task. A. Subjects were asked to move a manipulandum between two points on a screen (red, yellow). B. Example baseline movements from one participant showing the pattern of trial-to-trial variability. C. The subjects were rewarded based on how well their movements reflected predefined shapes (two shapes were used in different experiments). The shapes were never made explicit to the subjects, making it a trial-and-error learning task. D. Schematic showing baseline variability projected into the space defined by the two shapes. The target shapes were chosen to make sure that, on average, task-relevant variability was higher for Shape 1. E. Average learning curves showing that subjects generally learned Shape 1 faster than Shape 2. F. Task-relevant variability is correlated with learning level both across tasks (different colors) and individuals (markers). Adapted from (Wu et al., 2014).