Structure learning in a sensorimotor association task

Abstract

Learning is often understood as an organism's gradual acquisition of the association between a given sensory stimulus and the correct motor response. Mathematically, this corresponds to regressing a mapping between the set of observations and the set of actions. Recently, however, it has been shown both in cognitive and motor neuroscience that humans are not only able to learn particular stimulus-response mappings, but are also able to extract abstract structural invariants that facilitate generalization to novel tasks. Here we show how such structure learning can enhance facilitation in a sensorimotor association task performed by human subjects. Using regression and reinforcement learning models we show that the observed facilitation cannot be explained by these basic models of learning stimulus-response associations. We show, however, that the observed data can be explained by a hierarchical Bayesian model that performs structure learning. In line with previous results from cognitive tasks, this suggests that hierarchical Bayesian inference might provide a common framework to explain both the learning of specific stimulus-response associations and the learning of abstract structures that are shared by different task environments.

Figure 1. Task description.

(A) Subjects had to learn a mapping from a 3×3 stimulus board to a 3×3 action board. The stimulus was presented by lighting up one of the nine squares. The subject then had to press one of the nine response buttons associated to that stimulus. (B) There were six possible mappings with four different structures (S1 to S4). The identity and the random structure comprised only one mapping each. The shift structure consisted of a right-shift and a left-shift mapping. The mirror structure consisted of a horizontal and vertical mirror mapping.

Figure 2. Numbers of trials required by subjects to learn the different mappings.

(A) The first group learned the right shift before the left shift and the horizontal mirror before the vertical mirror. (B) The second group learned the two versions of the shift and mirror mappings in reverse order. Each group had 10 subjects. Statistical comparisons between the different mappings in each group can be found in Tables 1 and 2, and comparisons between the groups in Table 3. ID = Identity mapping. RS = Right shift mapping. LS = Left shift mapping. HM = Horizontal mirror mapping. VM = Vertical mirror mapping. RND = Random mapping.

Figure 3. Relative facilitation of learning.

(A) Mappings with structural constraints were learned much faster than the random mapping. (B) Learning the third (fifth) mapping was facilitated in both groups compared to learning the second (fourth) mapping. Shown are the medians and the lower and upper quartiles of the trial ratios of all subjects and the average has been taken over both groups.

Figure 4. Modelling of the facilitation effect.

(A) The experimental data shows a strong facilitation of learning a structured mapping (right-shift or left-shift) compared to a random mapping (RND). In addition, there is also a strong facilitation from learning the first instance of a shift mapping to learning the second instance. (B) The feed-forward neural network (NN) and the reinforcement learning (RL) model show no facilitation effects. The non-hierarchical Bayesian model shows a facilitation effect for the structured mappings if the prior probabilities of these mappings are elevated. The structure learning (SL) Bayes model shows both facilitation effects, because by learning the first mapping the posterior over structures assigns more probability to all other mappings with the same structure. All plots show median values, for the model these were computed over 100 simulation runs.

Figure 5. Graphical Model of the non-hierarchical and the hierarchical Bayesian model.

In the non-hierarchical model the observations provide evidence for each hypothesis separately. In the hierarchical model the observations not only provide evidence for the hypotheses, but also for the different structures (which in turn might shift some evidence to structure-compatible hypotheses).

Figure 6. Trial-by-trial evolution of learning.

For the experimental data we averaged over subjects to compute the probability that the correct action was chosen on the basis of the fraction of subjects that chose the correct action in each trial. For the model we determined the probability of choosing the correct action by computing the probability of choosing the correct action given the action and observation stream of each subject and again averaged over subjects. All curves were smoothed with a Savitzky-Golay-Filter of polynomial order 1 and length 11.

Figure 7. Forgetting errors in learning the different mappings.

Subjects committed two kinds of errors that involved forgetting. The first kind of error (leftside panels) occurs when subjects repeat a wrong response to a stimulus that they had already seen. The second kind of error (rightside panels) occurs when subjects had already pressed the correct button once, but later on seem to have forgotten this correct response and pressed a different button when once more confronted with the same stimulus. The upper panels show the total number of errors committed by subjects when learning the different mappings. The middle panels show the probability of an error occurring in each trial following the first trial of a new mapping (averaged over all subjects and mappings, in red all false button presses, in blue the two specific kinds of error). The lower panels show the proportion of errors that can be explained by stimulus-response patterns consistent with the previously learned structure (averaged over all subjects and mappings, in red proportion of all false button presses that can be explained by previous structure, in blue the proportion of the two specific kinds of error that can be explained by previous structure). The frequency histograms were smoothed over 50 trial windows by moving average.

Figure 8. Facilitation effect in the absence of error trials.

(A,B) Number of trials required by each subject to learn the mappings when disregarding all the error trials. (C,D) In the absence of error trials the facilitation effects remain all significant (p<0.02, Wilcoxon signed rank test).