Reverse Replay of Hippocampal Place Cells Is Uniquely Modulated by Changing Reward

Abstract

Hippocampal replays are episodes of sequential place cell activity during sharp-wave ripple oscillations (SWRs). Conflicting hypotheses implicate awake replay in learning from reward and in memory retrieval for decision making. Further, awake replays can be forward, in the same order as experienced, or reverse, in the opposite order. However, while the presence or absence of reward has been reported to modulate SWR rate, the effect of reward changes on replay, and on replay direction in particular, has not been examined. Here we report divergence in the response of forward and reverse replays to changing reward. While both classes of replays were observed at reward locations, only reverse replays increased their rate at increased reward or decreased their rate at decreased reward, while forward replays were unchanged. These data demonstrate a unique relationship between reverse replay and reward processing and point to a functional distinction between different directions of replay. VIDEO ABSTRACT.

Figure 1. Experimental design and place cell decoding

(A) Schematic of an experimental session. Reward was increased to 4× baseline at one end of the track in epoch 2 of Experiment One, and decreased to 0× in Experiment Two. See Figure S1 for running behavior. (B) Ripple (150–250Hz) filtered local field potential (top) and Bayesian decoding of the associated spike train during behavior (bottom). Actual position of rat overlaid in cyan. (C) Four example SWRs (top) with associated replay (bottom) which occurred at the ends of the track during the behavioral episode shown in B within the indicated time windows. See also Figure S1 and Table S1.

Figure 2. SWRs and replays are increased at 4× reward

(A) SWR occurrence during the first 20s of each stopping period over 15 laps, summed across all sessions. Color bar indicates number of SWRs. (B) Difference in SWR rate between ends of the track over the first 20s of each stopping period (mean ± SEM, n=maximum of 467 stopping periods in the equal reward condition and 217 in the unequal reward condition. Note that this number decreases over stopping period due to variability of time spent at the reward well). (C) Percent difference in SWR rate from unchanged to increased reward end of track in the equal and unequal reward conditions (mean ± 95% confidence interval). (D) Replay occurrence, as shown in A. (E) Difference in replay rate between ends of the track, as shown in B. (F) Difference in replay rate between ends of the track, as shown in C. *** indicates p < 0.001. See also Figures S2, and S4, and Table S2.

Figure 3. Reverse but not forward replays encode increase in reward

(A) Unidirectional place fields of 118 simultaneously recorded units in CA1. “Up” direction (left) and “down” direction (right) fields, sorted by “up” field centers (top) or “down” field centers (bottom). (B) Four example replays decoded using bi-directional fields (top), “up” directional fields (middle) and “down” directional fields (bottom). Replays were assigned the following identities, from left to right: forward replay moving up the track, forward replay moving down the track, reverse replay moving up the track, and reverse replay moving down the track. (C) Forward replay occurrence in the first 20s of each stopping period over 15 laps, summed across all sessions. Color bar indicates number of replays. (D) Difference in forward replay rate between ends of the track in the first 20s of each stopping period (mean ± SEM, as in Figure 2B). (E) Percent difference in forward replay rate from unchanged to increased reward end of track in the equal and unequal reward conditions (mean ± 95% confidence interval). (F) Reverse replay occurrence as shown in C. (G) Difference in reverse replay rate between ends of the track, as shown in D. (H) Difference in reverse replay rate between ends of the track, as shown in E. *** indicates p < 0.001. See also Figures S2–S5 and Table S2.

Figure 4. SWRs and replays are diminished in absence of reward

(A) SWR occurrence in the first 20s of each stopping period over 15 laps, summed across all sessions. Color bar indicates number of SWRs. (B) Difference in SWR rate between ends of the track in the first 20s of each stopping period (mean ± SEM, n=a maximum of 580 stopping periods in the equal reward condition and 230 in the unequal reward condition). (C) Percent difference in SWR rate from unchanged to decreased reward end of track in the equal and unequal reward conditions (mean ± 95% confidence interval). (D) Occurrence of replays as shown in A. (E) Difference in replay rate between ends of the track, as shown in B. (F) Difference in replay rate between ends of the track, as shown in C. ** indicates p < 0.01. *** indicates p < 0.001. See also Figures S2 and S4, and Table S3.

Figure 5. Forward and reverse replays at decreased reward

(A) Forward replay occurrence in the first 20s of each stopping period over 15 laps, summed across all sessions. Color bar indicates number of replays. (B) Difference in forward replay rate between ends of the track in the first 20s of each stopping periods (mean ± SEM, as in Figure 4C). (C) Percent difference in forward rate from unchanged to decreased reward end of track in the equal and unequal reward conditions (mean ± 95% confidence interval). (D) Occurrence of reverse replays as shown in A. (E) Difference in reverse replay rate between ends of the track, as shown in B. (F) Difference in reverse replay rate between ends of the track, as shown in C. * indicates p < 0.05. See also Figures S2–S4 and S6, and Table S3.

Figure 6. Comparison of forward and reverse replay

(A) Forward replay rate and reverse replay rate in epochs 1–3 in Experiment One (mean ± 95% confidence interval). (B) Forward replay rate and reverse replay rate in epochs 1–3 in Experiment Two (mean ± 95% confidence interval). * indicates p<0.05, ** indicates p<0.01, *** indicates p<0.001 adjusted for multiple comparisons. See also Table S4.

Figure 7. Changes in forward and reverse replay consistently code for relative increase or decrease in reward

(A) Schematic of relative increasing and decreasing phases of Experiments One and Two. (B) Set of coefficients from the linear model describing differences between forward and reverse replays, ends of track, and epochs in the increasing phase of both experiments (Experiment One, epoch 1 to epoch 2, and Experiment Two, epoch 2 to epoch 3). (C) Bootstrapped distribution of SSDs of Experiments One and Two, increasing phase. Red line represents data test statistic. (D) Set of coefficients from the linear model describing differences between forward and reverse replays, ends of track, and epochs in the decreasing phase of both experiments (Experiment One, epoch 2 to epoch 3, and Experiment Two, epoch 1 to epoch 2). (E) Bootstrapped distribution of SSDs of Experiments One and Two, decreasing phase. Red line represents data test statistic. See also Figures S4 and S7.