A Hippocampal Model for Behavioral Time Acquisition and Fast Bidirectional Replay of Spatio-Temporal Memory Sequences

Abstract

The hippocampus is known to play a crucial role in the formation of long-term memory. For this, fast replays of previously experienced activities during sleep or after reward experiences are believed to be crucial. But how such replays are generated is still completely unclear. In this paper we propose a possible mechanism for this: we present a model that can store experienced trajectories on a behavioral timescale after a single run, and can subsequently bidirectionally replay such trajectories, thereby omitting any specifics of the previous behavior like speed, etc, but allowing repetitions of events, even with different subsequent events. Our solution builds on well-known concepts, one-shot learning and synfire chains, enhancing them by additional mechanisms using global inhibition and disinhibition. For replays our approach relies on dendritic spikes and cholinergic modulation, as supported by experimental data. We also hypothesize a functional role of disinhibition as a pacemaker during behavioral time.

Keywords: cholinergic modulation; dendritic spikes; disinhibition; hippocampal replays; one-shot learning; place cells; reverse replays; synfire chains.

Figure 1

Robust replay of place cell activity. (A) An animal walks along a path, containing a loop in a 2-dimensional environment. (B) Locations in this path are associated to specific populations of CA3 place cells which fire at high rates if the animal is at said location. In this particular scenario, each position is associated with an ensemble of 100 place cells. Concomitantly, the sequence cells fire following a predefined sequential activity pattern that is explained in detail in section 2.2. Hebbian-like synaptic plasticity connects the sequence cells to the corresponding place cell ensembles. (C) The sequence cell dynamics allow for place cell activity replay on a much shorter timescale and independent of the animal's stopping times and locations. (D) 15 s simulation of place cell activity as the animal walks through the track in (A); a rate plot of the place cell ensembles corresponding to each position the animal walks through is shown; the x axis shows time in seconds, the left y axis shows the track positions corresponding to the place cell populations and the right y axis shows the firing rate of each place cell ensemble corresponding to a particular track position. Note that in addition to the loop in the path, the animal occasionally stays arbitrarily longer at particular locations. (E) Fast replay of the path taken by the animal in less than 100ms; a raster plot of place cell activity is shown; the x axis shows time in milliseconds, the left y axis shows the track positions corresponding to the place cell populations. (F) Reverse replay of the path taken by the animal; a raster plot of place cell is shown; the axis are the same as in (E). Note that each weight in the spiking network is multiplied by a gaussian multiplicative noise.

Figure 2

Sequence encoding modules. (A) Illustration of a sequence encoding module architecture consisting of 3 bistable units B_i, B_i+1, B_i+2, the competitive inhibitory population I_C and the gating inhibitory input I_G. The inhibitory unit I_C prevents two bistable units B_j from having high rate at the same time, whereas the gating inhibition I_G is the ”break” that prevents activity from propagating from B_j to B_j+1. This break is externally controlled, by some mechanism that recognizes when the animal changes its location. (B) Evolution of the rates of 2 abstract rate units B_i, I_C and I_G, disposed according to the sequence encoding architecture in (A). (C) General illustration of the states the network goes through over time. (i,vi) correspond to stable states in the network while I_G is activated. Inactivating I_G induces a dynamical process that brings the network from state (ii) into state (v). This dynamical process is depicted in further detail below with (iii,iv). In (iii), due to inactivation of I_G, the activity in B_i+1 picks up, driven from the activity B_i. As shown in (iv), once both B_i and B_i+1 are active, they together activate I_C, which gives strong inhibitory input to B_i and B_i+1 and (in v) forces B_i to become inactive, while the recurrent activity permits B_i+1 to recover and stay active. (D) Spiking network of sequence cells. Above, the x-axis shows 1.2 s of simulated activity. The left y-axis shows the indices of the cells in each sequence ensemble, and a raster plot is given. The right y-axis shows the rate of each sequence ensemble. Below, the period where I_G is inactivated is emphasized. During this period, the stability of the system is broken and the dynamics described above take place, leading ultimately to B_i+1 overtaking B_i as the active sequence ensemble. (E) Analogous to (D), but we now purposedly enforce a longer time out of the global inhibitory mechanism. Observe that the system still functions much like in (D), so the period where I_G is turned off is flexible.

Figure 3

Replay mode of the sequence encoding module. (A) Synfire chain like synchronous activation of three consecutive bistable units B₁, B₂, B₃. Synchrony triggers dendritic spikes in the next bistable unit, which in turn synchronously activate it. Activity moves in a single direction due to neurons entering a refractory period. (B) Nonlinear effect of a dendritic spike on the membrane potential at the soma of a neuron. There is a latency between the moment the dendrite triggers the dendritic spike and its effect on the soma. All dendritic spikes produce the same stereotypical current pulse effect in the soma. (C) Raster plot of the sequence encoding module in the replay mode, without any dendritic spikes, and showing different levels of cholinergic effect (for ACh factor of 5, only the first five spikes are shown). Without any cholinergic modulation (red), or with the cholinergic factor used in the rest of the paper (black), the sequence encoding module does produce a replay of the path, but on a much slower timescale. Each transition, in the same way as during encoding, takes between 50 and 100 ms. If one increases the cholinergic effect beyond what is reported in Hasselmo and Schnell (1994) and Hasselmo et al. (1995), then as shown in the plot (blue), the transitions between consecutive ensembles is around 15 ms in the upper range of what is often reported in the literature (Lee and Wilson, ; Diba and Buzsáki, 2007). We emphasize that only the first five spikes are shown in the blue plot. Due to the fivefold increase in excitatory weights and no counterbalance by inhibition, the network becomes too excitable and the neurons spike at very high rates. Still, the blue plot indicates that it would be possible to produce fast replays if the cholinergic effect would be two times or more of what has been reported in the literature, even without using dendritic spikes. On the other hand, reverse replays rely crucially on dendritic spikes and cholinergic modulatory effect and do not happen without either. (D) Raster plot of the sequence encoding module in the replay mode, now with dendritic spikes. Dendritic spikes increase the speed of replays by two orders of magnitude, making it in line with what is observed during sharp-wave ripples.

Figure 4

Spiking network. (A) Illustration of a bistable unit consisting of both an excitatory and an inhibitory population. (B) On the left, the gain function of excitatory (red) and inhibitory(teal) neurons inside a bistable unit B_i is depicted. As shown in the center, if I_G is turned on, then the shape of the gain function produces two stable states, one at 0Hz and another at approximately 50Hz. The stable states are shown on the right. (C) Analogous to (B), but now with I_G turned off. As seen on the left, the gain function shape changes when receiving input from the previous pattern and the stable state at 0Hz ceases to exist. The network always settles for the only stable point around 50 Hz, a fact which is shown on the right.

Figure 5

Robustness to parameter variations. (A) Raster plot of the encoding mode of the sequence encoding module with 6 patterns. The size of ensembles B_i and I_C are doubled, while the connection probability is halved (alternatively, one can half all corresponding weight parameters). This shows that the size of each ensemble is not important for the model to work. (B) Analogous to (A) but now the replay mode is shown. Again each ensemble size is doubled. (C) Raster plot of the encoding mode of the sequence encoding module, with an alternative parameter set (see Table 6). This shows that the choice of parameters we used is not unique. Instead, as long as the conditions on the gain functions are satisfied, there is a large range of weight parameters where the model performs as expected. (D) Raster plot of the replay mode using the alternative parameter set (see Table 6).