The Computational and Neural Bases of Context-Dependent Learning

Abstract

Flexible behavior requires the creation, updating, and expression of memories to depend on context. While the neural underpinnings of each of these processes have been intensively studied, recent advances in computational modeling revealed a key challenge in context-dependent learning that had been largely ignored previously: Under naturalistic conditions, context is typically uncertain, necessitating contextual inference. We review a theoretical approach to formalizing context-dependent learning in the face of contextual uncertainty and the core computations it requires. We show how this approach begins to organize a large body of disparate experimental observations, from multiple levels of brain organization (including circuits, systems, and behavior) and multiple brain regions (most prominently the prefrontal cortex, the hippocampus, and motor cortices), into a coherent framework. We argue that contextual inference may also be key to understanding continual learning in the brain. This theory-driven perspective places contextual inference as a core component of learning.

Keywords: Bayesian inference; context-dependent learning; continual learning; hippocampus; learning; memory; motor cortex; neural computation; prefrontal cortex; thalamus.

Figure 1. A normative model of context-dependent learning.

a. Generative model, showing two consecutive time steps, t − 1 and t. Context, c_t, evolves over time (color represents the identity of the active context). Each context j is associated with a set of contingencies, $x_t^{\left(j\right)}$, that may also evolve in time, independent of the contingencies of the other contexts. (Only two contexts are shown for simplicity; in general, the number of contexts and sets of contingencies can be unbounded). The contingencies of the currently active context (filled vs. empty circle respectively gating black vs. gray arrows) determine what sensory cue is received, q_t (pink), and how the agent’s action, a_t (green), change its state, s_t (orange), and lead to sensory feedback, r_t (purple). In general, the contingencies of the active context can also affect the next context transition (not shown for clarity), and states may not be directly observable. b. Inference, showing two consecutive time steps. The agent infers contexts (top, inferred posterior distributions shown as histograms) and context-specific contingencies (middle, inferred posterior distributions in a multi-dimensional space of contingencies illustrated with covariance ellipses, dimensions [dim.] 1 and 2 shown) based on observed states, sensory cues, sensory feedback, and its own actions (bottom, colors as in a). Contexts are color coded in top and middle as in panel a. Note that the probability that a yet-unseen, novel context with ‘default’ contingencies is encountered is also always computed (gray). Columns 1 and 2: the context posterior at time t (column 1, top) is computed by fusing prior expectations (propagated context posterior from time t − 1, not shown) with information about the current observations (column 1, bottom). This posterior determines the degree to which information from the current observations is used to update the inferred context-specific contingencies at time t + 1 (column 2, middle, memory updating, pale blue distribution updated to bright blue distribution) and whether a new memory is instantiated (column 2, middle, memory creation, the red posterior over contingencies updated from the gray posterior over contingencies of a novel context). Column 3: for each represented context, the expected values of actions (curves in context-specific colors) are computed based on the inferred contingencies of that context. Column 4: the final expected values of actions (green curve) are the weighted sum of their context-specific values, with the weighting determined by the posterior probabilities of the corresponding contexts (memory expression). The action with the highest final expected value is chosen (vertical green line; potentially with some decision or motor noise, not shown). Note that memory creation, expression and updating can all occur on the same time step, but they are shown here on separate time steps to avoid visual clutter.

Figure 2. Neural basis of contextual learning.

a. Prefrontal cortex encodes context probabilities. Participants learned strategies that involved choosing an action (correct button press) in response to a state observation (number displayed on a screen). The state-action contingencies occasionally switched between three possibilities, necessitating different context-specific strategies. Brain activations (fMRI) that correlated with the probability (strategy ‘reliability’, derived from a behavioral model) of the context associated with either the participants’ current strategy (magenta), or with alternative strategies (cyan). Reproduced from Donoso et al. 2014. b. Hippocampal activity samples from context-specific maps under context uncertainty. Hippocampal activity was recorded as rats navigated one of two spatial environments. Occasionally, animals would be “teleported” between the environments by instantaneously switching sensory cues. Left: time series (columns show successive theta cycles) of correlations between current neural population activity and reference activity associated with each location in environment A (top, red) and environment B (bottom, blue). A teleportation from environment A to B took place shortly before this series. The correlation shows evidence of spontaneous flickering between the hippocampal place-field maps associated with each environment despite no change in the environment during this period (green cross shows current location). Right: histogram shows a temporary increase in the number of flickers after a change in the environment. Reproduced from Jezek et al. 2011. c. Hippocampal representation of a prior distribution over contexts. Hippocampal activity was recorded as mice navigated through virtual reality environments that could morph between two extremes by continuously varying the frequency of a sinusoidal grating on the walls, thus giving rise to a continuum of contexts. Mice experienced one of two different distributions of contexts, characterized by different distributions of the morphs (“rare” vs. “frequent” morph distributions, respectively red vs. blue thick lines). Thin lines show the prior reconstructed from the hippocampal activity of individual mice exposed to the rare (left) and frequent (right) distributions. The prior approximated the true distribution for each condition. Reproduced from Plitt & Giocomo 2021. d. Context-dependent motor learning. Nonhuman primates reached to targets under two possible force perturbations applied to the hand (forces applied by a robotic interface that act in clockwise, CW, or counter-clockwise directions, CCW) or unperturbed. Neural state space show preparatory activity (after dimensionality reduction) in dorsal premotor and primary motor cortex for movements to the same 12 targets (points) under the three possible contexts: before learning (black), CW perturbation (red), and CCW (blue) perturbations. Projections of the neural state space are shown on the ‘floor’ and ‘wall’. Figure plotted from data in Sun et al. 2022. e. Neural representation of extinction in the mouse amygdala during fear conditioning. Change in a distance measure between the neural representation of the CS during training (conditioning and extinction) and the pre-learning representation of the US (red) or CS (blue). Negative values mean the representations become more similar. Reproduced from Grewe et al. 2017.