Rewarded outcomes enhance reactivation of experience in the hippocampus

Abstract

Remembering experiences that lead to reward is essential for survival. The hippocampus is required for forming and storing memories of events and places, but the mechanisms that associate specific experiences with rewarding outcomes are not understood. Event memory storage is thought to depend on the reactivation of previous experiences during hippocampal sharp wave ripples (SWRs). We used a sequence switching task that allowed us to examine the interaction between SWRs and reward. We compared SWR activity after animals traversed spatial trajectories and either received or did not receive a reward. Here, we show that rat hippocampal CA3 principal cells are significantly more active during SWRs following receipt of reward. This SWR activity was further enhanced during learning and reactivated coherent elements of the paths associated with the reward location. This enhanced reactivation in response to reward could be a mechanism to bind rewarding outcomes to the experiences that precede them.

Figure 1. Task design and behavioral performance

a, Overhead view of the behavioral apparatus with reward sequences indicated by colored arrows (Sequence 1 (S1) in blue and Sequence 2 (S2) in red). Brown circles indicate the location of food wells; reward was delivered in arms B–E. Arms are 7.25 cm wide. Scale bar is 50 cm. b, Twenty trial moving average of correct responses for one animal when switching between performing S1 (blue) and S2 (red). Background color indicates which sequence was rewarded. Black lines separate recording days. Top shows all sequence switching days, bottom shows first day of switching (day 6). Chance performance on this task is 0.2 as there are 5 arms the animal can choose from when leaving an arm. c, Examples of individual trials at the end of session one when S1 was rewarded (far left), the beginning of session 2 just after the reward contingencies change to reward S2 (middle left), the middle of session 2 (middle right), and the end of session 2 (far right). The animal’s path through space is shown in black and the rewarded sequence arms are highlighted (S1 in blue and S2 in red). The arrow indicates direction. A brown filled circle indicates that the animal received chocolate at the end of the trajectory while a “Ø” indicates no reward was delivered. In the first two columns the animal performed the same sequence of trajectories but was rewarded differently. In session 1 all of the trajectories shown were rewarded as they are the correct performance of S1 (far left). Then at the beginning of session 2 (middle left), the animal was placed in the center arm of the newly rewarded sequence (S2) but performed the previously rewarded sequence (S1). In this case, only the third trajectory was rewarded (it is a correct return to the home arm of S2) while the other trajectories were unrewarded.

Figure 2. Enhanced SWR activity following reward

a, Examples of a single unrewarded (left) and rewarded (right) trial including times at the well (black horizontal lines) and the preceding run period. Trials are from the same epoch and contain the same cells. Spikes are shown in black. Spikes that occurred during SWRs when animals were stopped at the well (grey bars on raster plot) appear in the grey bar. SWRs were detected using the simultaneously recorded EEG filtered at 150–250Hz (dark grey traces above raster plot, top: unfiltered, bottom: filtered 150–250Hz). The top traces show the SWR in unfiltered and filtered EEG at higher magnification. Activation probability per trial (b), wSWR rate (c), and activation probability per wSWR (d) for cells with place fields on the track when animals were stopped at the well in rewarded and unrewarded trials. Cumulative distribution of the mean number of spikes per wSWR (e), the mean firing rate (f), and the proportion of cells active (g) during wSWRs on rewarded (grey) and unrewarded trials (black). h, Activation probability per trial when rewarded trials were truncated to match the duration of immobility on unrewarded trials. Error bars represent standard errors. Bar graphs include only times when animals were stopped at the well. *** indicates p < 10⁻⁴.

Figure 3. SWR activity following reward is further enhanced during learning

a, wSWR on rewarded trials before (‘pre’) and after (‘post’) reward contingencies changed for the first time (e.g. before and after the first exposure to S2). b, Activation probability per wSWR and c, proportion of cells active per wSWR in S1 (1^st and 3^rd bars) or S2 (2^nd and 4^th bars) on the first and third day of switching between sequences. All data include only rewarded trials when animals were stopped at the well. Error bars represent standard errors. *** indicates p < 10⁻⁵, ** indicates p < 0.005, and * indicates p < 0.05.

Figure 4. Reactivation of single cells during SWRs following reward

a, Proportion of cells active during wSWRs that are also active on the preceding run to the well (past) or the subsequent run from the well (future). b, Activation probability per wSWR for cells with (dark grey) and without (light grey) place fields on the track. c, Within cell differences in activation probability per wSWR between rewarded and unrewarded trials for cells with (dark grey) and without (light grey) place fields on the track. Activation probability for one cell on unrewarded trials was subtracted from that on rewarded trials. Positive differences indicate that rewarded trials had a greater activation probability than unrewarded trials. d, Summed firing rate maps of run period spiking of all cells that fired during wSWRs at the well indicated by the brown circle. Spikes from all simultaneously recorded cells are shown together for each of 3 different epochs, one from each animal. Each spike for each cell contributed equally, no attempt was made to normalize each cell by their firing rate. The plots include 7 (left), 4 (middle), and 14 (right) cells. e, Location of peak occupancy normalized firing rate during run periods of all cells (grey) and cells that fired in ripples at the end of the trajectory (black) on rewarded trials. The distributions of peak firing locations on each trajectory were averaged together. Note that the two curves are almost entirely overlapping. Error bars represent standard errors. *** indicates p < 10⁻⁵.

Figure 5. Structured reactivation of pairs of cells during SWRs following reward

a, Coactivation probability per wSWR on rewarded and unrewarded trials. b, Coactivity z-score per wSWR versus place field overlap on rewarded trials. Coactivity z-score was binned according to the cell pairs’ place field overlap: no overlap, greater than 0 but less than 0.2, 0.2 to 0.4, and 0.4 to 0.6 overlap. Because many cells never fire in ripples on unrewarded trials, there were too few coactive pairs to perform this analysis on unrewarded trials. c, Coactivation among cell pairs per 100 msec bin during the run period of rewarded and unrewarded trials. d, Scatter plot of distance between place field peaks versus inter-cell inter-spike interval (ISI) during wSWRs. Each point is one inter-cell ISI during wSWR plotted against the distance between the cells’ peaks on that pass. Only spikes that occurred during wSWRs when animals stopped at the well were included in measures of wSWR inter-cell ISI. A singer ISI could extend over multiple SWRs, as previous work has shown coherent reactivation across SWRs (Davidson et al., 2009). Previous quantifications of replay have found replay in a subset of SWRs, and the R² values seen here would be expected to reflect the combination of events with replay and events where replay was absent. Error bars represent standard errors. * indicated p < 0.01, ** indicates p < 0.0001, *** indicates p < 10⁻¹⁰.

Figure 6. Comparison of SWR activity across behavioral conditions

a, Activation probability for the same behavioral sequence with different rewards. Left: schematic of sequence rewarded (highlighted on track, S1 in blue and S2 in red) and sequence performed (arrows below track) for data shown at right. Right: activation probability per wSWR when animals accurately performed S1 and reward was omitted (S2 rewarded, S2r/S1p) or delivered (S1 rewarded, S1r/S1p). b, Activation probability per wSWR during the first rewarded exposure to S2, when reward was unexpected based on the previous history of rewards, and unrewarded trials in all arms. Only times when animals were stopped at the well were included. *** indicates p < 10⁻⁵.