Pupil-linked arousal signals track the temporal organization of events in memory

Abstract

Everyday life unfolds continuously, yet we tend to remember past experiences as discrete event sequences or episodes. Although this phenomenon has been well documented, the neuromechanisms that support the transformation of continuous experience into distinct and memorable episodes remain unknown. Here, we show that changes in context, or event boundaries, elicit a burst of autonomic arousal, as indexed by pupil dilation. Event boundaries also lead to the segmentation of adjacent episodes in later memory, evidenced by changes in memory for the temporal duration, order, and perceptual details of recent event sequences. These subjective and objective changes in temporal memory are also related to distinct temporal features of pupil dilations to boundaries as well as to the temporal stability of more prolonged pupil-linked arousal states. Collectively, our findings suggest that pupil measures reflect both stability and change in ongoing mental context representations, which in turn shape the temporal structure of memory.

Fig. 1. Auditory event boundary paradigm.

Participants studied lists of 32 everyday objects and had to indicate whether each item would more likely be encountered in an indoor or outdoor setting. The surrounding context was manipulated by playing a simple tone in either participants’ left or right ear prior to viewing each image, which indicated to participants which hand they should use to make their subsequent indoor/outdoor judgment. After eight successive items, the tone switched to the other ear and changed in pitch. These tone switches served as event boundaries, which parsed each continuous 32-item sequence into four discrete auditory events. After a short distractor task, participants performed three different memory tests in two separate blocks of trials: the first block included two different temporal memory tests and the second block included a source memory test. In the temporal memory block, participants first had to indicate which of the two presented items had appeared more recently in the prior sequence. Second, they had to rate the temporal distance between these items, ranging from ‘very close’ to ‘very far’ apart in the sequence. After being tested on item pairs, participants performed an auditory source memory test for all of the remaining items that were not shown during the temporal memory tests. In this source memory test, participants were shown individual items and had to indicate whether each item had been paired with a tone in their left ear or their right ear. The hand button-press icon was made by Freepik from www.flaticon.com.

Fig. 2. Individuals are slower to respond to items appearing just after a tone switch, or event boundary, compared to other items in an event sequence.

Values represent average response times (RTs) for the indoor/outdoor item judgments during sequence encoding. Colored boxplots represent 25th–75th percentiles of the data, the center line the median, and the error bars the s.e.m. Overlaid dots represent individual participants (Experiment 1: n = 34; Experiment 2: n = 35; Experiment 3: n = 30). Two-tailed paired pairwise t-tests were performed to test for significant differences between judgment response times for boundary items, first items, last items, and same-context items. These t-tests were planned, so no adjustments were made for multiple comparisons. The results for the boundary compared to same-context trial comparisons from the first experiment directly replicated two follow-up experiments. Source data are provided as a Source Data file.

Fig. 3. Tone switches, or event boundaries, embedded within item sequences lead to impaired order memory and expanded retrospective estimates of temporal distance between item pairs spanning those boundaries.

a Values represent average temporal distance ratings for item pairs from the object sequences. During this temporal memory test, participants rated how far apart the item pairs had appeared in the prior sequence, with choices ranging from ‘very close’ to ‘very far’. The ratings were then converted to a scale ranging from 1 to 4 and averaged together, such that higher values on the y-axis reflect more expanded retrospective estimates of temporal distance between item pairs. b Values represent average temporal order memory accuracy for item pairs from the object sequences. During this temporal order memory test, participants had to decide which of two items had appeared later (i.e., more recently) in the previous sequence of images. ‘Lag’ refers to the number of intervening items that had appeared between the to-be-tested item pairs at encoding. For both panels, colored boxplots represent 25th–75th percentiles of the data, the center line—the median, and the error bars—the s.e.m. Overlaid dots represent individual participants (Experiment 1: n = 34; Experiment 2: n = 35; Experiment 3: n = 30). Two-tailed paired t-tests were performed to test for significant differences between temporal memory outcomes for boundary pairs compared to same-context pairs. These t-tests were planned, so no adjustments were made for multiple comparisons. The results from the first experiment were replicated in two follow-up experiments. Source data are provided as a Source Data file.

Fig. 4. Tone switches lead to better source memory for individual items and their accompanying sounds.

During this source memory test, participants had to indicate whether each individually presented item had been paired with a tone played in their left or right ear during encoding. ‘Last Item’ refers to the last image presented in each 32-item list, whereas ‘First Item’ refers to the first image presented in each 32-item list. ‘Boundary Item’ refers to the first object appearing after a tone switch, or event boundary. Colored boxplots represent 25th–75th percentiles of the data, the center line—the median, and the error bars—the s.e.m. Overlaid dots represent individual participants (Experiment 1: n = 34; Experiment 2: n = 35; Experiment 3: n = 30). A repeated-measures ANOVA was performed to test for differences in source memory accuracy by the four different item types. Displayed p-values reflect the results of follow-up, two-tailed Bonferroni-corrected pairwise comparisons. The results for the boundary compared to same-context trial comparisons from the first experiment directly replicated two follow-up experiments. Source data are provided as a Source Data file.

Fig. 5. Event boundaries modulate temporal characteristics of pupil dilation.

a The effects of boundary (tone switches; blue) versus same-context tones (repeated; gray) on pupil dilation. Shaded areas represent s.e.m. b Average fluctuations in pupil diameter across the encoding sequence, averaged across all blocks, experiments, and participants. The gray shaded area represents s.e.m. c Temporal features of tone-evoked pupil dilation identified by a temporal principal component analysis (PCA). The PCA revealed four significant features of pupil dilation that had distinct shapes over time. Component loadings reflect “raw” values from the rotated solution, so are on the same scale as the original inputs. Gold dashed lines signify the onsets of the tones and their subsequent images. d The boundary tones (blue colors) versus same-context tones (gray) differentially modulated loading scores for the first three pupil components but not the fourth component. Colored boxplots represent 25th–75th percentiles of the data, the center line the median, and the error bars the s.e.m. Overlaid dots represent individual participants (Experiment 2: n = 35; Experiment 3: n = 30). Two-tailed paired pairwise t-tests were performed to test for significant differences between factor-loading values for boundary trials compared to same-context trials. No adjustments were made for multiple comparisons. Source data are provided as a Source Data file.

Fig. 6. Temporal characteristics of pupil dilation evoked by tones and their subsequent images reflect different motor and anticipatory aspects of the sequence learning task.

a A temporal principal component analysis (PCA) on the pupil dilation data revealed four pupil components that had distinct shapes across time. b Two follow-up PCAs separated by condition help illustrate how loading on these components differed between event boundary trials and same-context trials. Vertical dashed lines signify the onsets of the tones and their subsequent images. In both conditions, most of the temporal characteristics of the pupil component loadings were qualitatively similar, except for the early-peaking component (component #4; turquoise). c To better illustrate these differences, only component four’s loading time-course is displayed. The plot reveals evidence of this early peaking component in response to the boundary tones, boundary images, and same-context images (red arrows). However, this component did not peak in response to same-context tones (red shaded area), which was the only stimulus type (tone or image) that did not require a motor response. d Only the slowly decreasing component (#3; sky blue) is highlighted to illustrate potential differences in its loading patterns over time. A peak in this component’s loadings is identifiable for each tone and image type (red arrows).

Fig. 7. Different temporal characteristics of boundary-triggered pupil dilation relate to subjective and objective aspects of temporal memory.

a Association between boundary-modulated loadings on the early-peaking pupil component (#4; turquoise) and boundary-related effects on retrospective estimates of temporal distance between item pairs. Higher values on the y-axis reflect more expanded estimates of average temporal distance for boundary-spanning pairs compared to same-context pairs. This pupil–memory relationship was significant across both eye-tracking studies (Experiments 2 and 3) and significant in Experiment 3 alone. b Association between boundary-modulated pupil loadings on the slowly decreasing pupil component (#3; sky blue) and boundary-related effects on temporal order memory between item pairs in Experiment 3. Lower values on the y-axis reflect worse average temporal order memory for boundary-spanning item pairs compared to same-context pairs. Gray shading in all three regression plots represents 95% confidence interval. Overlaid black dots represent individual participants (Experiment 2: n = 35; Experiment 3: n = 30). p-values were derived from two-tailed tests, and no corrections were made for multiple comparisons. Source data are provided as a Source Data file.