Learning, fast and slow

Abstract

Animals can learn efficiently from a single experience and change their future behavior in response. However, in other instances, animals learn very slowly, requiring thousands of experiences. Here, I survey tasks involving fast and slow learning and consider some hypotheses for what differentiates the underlying neural mechanisms. It has been proposed that fast learning relies on neural representations that favor efficient Hebbian modification of synapses. These efficient representations may be encoded in the genome, resulting in a repertoire of fast learning that differs across species. Alternatively, the required neural representations may be acquired from experience through a slow process of unsupervised learning from the environment.

Figure 1:. Learning rates.

The complexity of various tasks plotted against the number of reinforcement trials required to learn them. Note the learning rates span 4 orders of magnitude. See text and Section A for literature and calculations.

Figure 2:. Pattern separation in higher dimensions.

(a) Here m different events (dots) are encoded by the firing rate of 2 sensory neurons (x₁ and x₂). The brain wants to classify those events into good (blue) and bad (red). In the original sensory representation that would require computing the complex region inside the dashed line. (b) After projecting the sensory data into a high-dimensional space, represented by N > m neurons (y₁, …, y_N), one can generally find a hyperplane (green) such that all the good points are on one side and the bad ones on the other. The projection from sensory signals x_j to the over-complete representation y_i can take the form $y_i=f\left({\sum }_{j=1}^2{w}_{ij}{x}_j\right)$ where w_ij are random synaptic weights and f is some nonlinear activation function.

Figure 3:. Associative learning and sparseness.

(a) A simple network to learn the mapping from a set of states onto a set of actions. Stimuli are represented by n neurons that are either active or inactive, s_i ∈ 0, 1. Similarly actions are represented by n neurons. During learning, the network is exposed to the k desired (state,action) pairs. The synapse from a state neuron to an action neuron is incremented if both pre- and post-synaptic neurons are active. (b) Recall of the stored association: A state vector is presented to the input and the output of the network is compared to the k possible action vectors; this shows the resulting similarity matrix. In this case the state and action vectors each have only m = 1 of n = 10 active neurons. The recall of actions associated with each state is perfect. (c) As in panel b, but each vector is represented by 3 active neurons. Note the extensive confusion from the intended mapping of states onto actions.

Figure 4:. Graphs that define behavioral tasks.

(a) A typical 2-alternative-forced-choice task, with states defined by stimuli s₁ and s₂ and actions of the animal a₁ and a₂. The square states terminate a trial with reward (green) or no reward (red). (b) A more complex task in which the animal starts in state s₁ and then navigates a binary tree by turning left (a₁) or right (a₂) or backing up to the previous state (a₀). Only one of the end points of the tree is rewarded. The task can be implemented as navigation in a binary maze.