Time for Memories

Abstract

The ability to store information about the past to dynamically predict and prepare for the future is among the most fundamental tasks the brain performs. To date, the problems of understanding how the brain stores and organizes information about the past (memory) and how the brain represents and processes temporal information for adaptive behavior have generally been studied as distinct cognitive functions. This Symposium explores the inherent link between memory and temporal cognition, as well as the potential shared neural mechanisms between them. We suggest that working memory and implicit timing are interconnected and may share overlapping neural mechanisms. Additionally, we explore how temporal structure is encoded in associative and episodic memory and, conversely, the influences of episodic memory on subsequent temporal anticipation and the perception of time. We suggest that neural sequences provide a general computational motif that contributes to timing and working memory, as well as the spatiotemporal coding and recall of episodes.

Figure 1.

A, Brain mechanism-based view of episodic memory: the hippocampus as a sequential multiplexed pointer. Indices that point to cortical modules for different inputs are sequenced by evolving hippocampal activity patterns, thus preserving the directed ordinal structure over which experience occurred. Semantic content resides in cortical modules that are concatenated by the hippocampus during both encoding and recall. Aud, Auditory; Olf, olfactory; Som, somatosensory; Vis, visual. Adapted with permission from Buzsáki and Tingley (2018). B, Sequential activation of neuronal assemblies in an episodic memory task. Middle, The rat was required to run in a running wheel for 15 s before choosing either the left or the right arm of the maze based on the remembered last arm choice. It obtained a water reward if it chose the opposite of the previously chosen arm. Color-coded dots represent spike occurrences of simultaneously recorded hippocampal neurons. Left, Normalized firing-rate profiles of neurons during wheel running, ordered by the latency of their peak firing rates during left trials. Each line indicates a single cell. Right, Normalized firing rates of the same neurons during right trials. An observer can infer the run duration (and distance) in the wheel as well as the future choice of the rat from the same sequential firing patterns of the neurons. Adapted with permission from Pastalkova et al. (2008). C, Time detection on single trials using a time prediction model fit from all other trials. In each time bin, elapsed time in the running wheel is inferred either from the population firing rate vector (red) or the firing phases of active cells with respect to the theta oscillation (purple). In each case, the prediction approximates well the true time (black). D, Error of time estimation from population vector of neuronal activity in 3 rats. Rat 1 had <50 recorded neurons; Rats 2 and 3 had >50 neurons. Note reliable estimation of running duration from neuronal activity. Adapted with permission from Itskov et al. (2011).

Figure 2.

A, Schematic of the dDMS task (left). Inverse efficiency (RT/accuracy) of human subjects on the dDMS task across Standard (cyan) and Reverse (orange) trials. The short and long delays correspond to the actual delay epochs. There was a significant main effect of Standard versus Reverse trial (n = 27, F_1,26 = 9.05, p < 0.01). B, Schematic of the RNN architecture and the inputs and target outputs for the dDMS task during the control and reverse conditions. C, Neurograms during the AA (top row) and BA (bottom row) conditions (A, red line above neurogram; B, green), sorted according to the peak time during the short (left) or long (right) delays (standard trials). The self-sorted neurograms (top left and bottom right) are cross-validated (average of even trials sorted on average of odd trials). The overlaid white and gray lines indicate the “motor” unit (right y axis) and “temporal expectation” output unit, respectively.

Figure 3.

Results showing flexible temporal anticipation for WM retrieval (A) and based on long-term memory (B). A, Task schematic and behavioral results in the study by van Ede et al. (2017). Participants encoded two randomly oriented lines presented for 250 ms on the left and right of fixation. Lines were colored light orange and blue, and their side was random. After a short (1250 ms) or long (2500 ms) delay, a central probe prompted participants to reproduce the orientation of one of the lines. Color of the probe handle represents the item to be reproduced. The main manipulation was that the probability of being probed about the orange or blue item varied over time. Items presented in one of the colors was more likely (80% probability) to be probed after the short delay, and items in the other color were more likely to be probed after the long delay. Schematic represents the case in which the orange item is likely to be probed earlier and the blue item to be probed late. Participants had unlimited time to activate the reproduction response by clicking the mouse but then had limited time (2500 ms) to complete the response, thus yielding a decision response time and an angular error for the response in each trial. Decision times indicated that reproduction responses started earlier when participants were probed about the temporally expected items (red placeholders) at both short and long intervals. Angular errors were also smaller when retrieving the temporally expected item (red placeholders). Note the flexible reprioritization of items, such that colored items yielded slower and less accurate responses when unexpectedly probed early but fast and accurate responses when probed later, when expected. B, Task schematic and results from Cravo et al. (2017, their Experiment 1). In a learning session, participants viewed scenes in which a placeholder cartoon bomb appeared after 1500 ms. For a given scene, the bomb changed color after either a short (800 ms) or long (2000 ms) interval. Participants made a speeded response if the target turned blue (80% of trials, go target) and withheld from responding if the target turned red (20%, no-go targets). Over five blocks, participants learned the implicit temporal associations between scenes and intervals, with responses becoming faster and more accurate over blocks. A temporal-orienting task was performed after learning. The interval between placeholder and target matched that in the learning task on the majority (67%) of the trials (valid memory cues) but was reversed in the remaining trials (33%, invalid memory cues). Response speed and sensitivity were significantly improved when targets occurred at their learned long-term memory intervals. Both reaction times and perceptual sensitivity were better for targets following Valid (V) than Invalid (I) memory cues. The grand-averaged CNV potential was localized over the frontal temporal scalp and was strongly modulated by the temporal association. The CNV developed earlier and was steeper after scenes associated with short placeholder-target delays (red line) compared with long placeholder-target delays (gray line).

Figure 4.

A, Ezzyat-DuBrow-Davachi paradigm for studying the effect of event boundaries on episodic memory. B, Memory for the correct temporal order of items is significantly better when judging within context pairs (right). When asked to judge whether pairs of items were close or far apart, participants were more likely to rate items separated by the same distance as close when the pair was within the same context, and as “far” across contexts (right). Adapted with permission from Ezzyat and Davachi (2011); DuBrow and Davachi (2013, 2014, 2016); Heusser et al. (2016); Clewett et al. (2019).