Ocular working memory signals are flexible to behavioral priority and subjective imagery strength

Abstract

The pupillary light response was long considered a brainstem reflex, outside of cognitive influence. However, newer findings indicate that pupil dilation (and eye movements) can reflect content held "in mind" with working memory (WM). These findings may reshape understanding of ocular and WM mechanisms, but it is unclear whether the signals are artifactual or functional to WM. Here, we ask whether peripheral and oculomotor WM signals are sensitive to the task-relevance or "attentional state" of WM content. During eye-tracking, human participants saw both dark and bright WM stimuli, then were retroactively cued to the item that would most likely be tested. Critically, we manipulated the attentional priority among items by varying the cue reliability across blocks. We confirmed previous findings that remembering darker items is associated with larger pupils (vs. brighter), and that gaze is biased toward cued item locations. Moreover, we discovered that pupil and eye movement responses were influenced differently by WM item relevance. Feature-specific pupillary effects emerged only for highly prioritized WM items but were eliminated when cues were less reliable, and pupil effects also increased with self-reported visual imagery strength. Conversely, gaze position consistently veered toward the cued item location, regardless of cue reliability. However, biased microsaccades occurred at a higher frequency when cues were more reliable, though only during a limited post-cue time window. Therefore, peripheral sensorimotor processing is sensitive to the task-relevance or functional state of internal WM content, but pupillary and eye movement WM signals show distinct profiles. These results highlight a potential role for early visual processing in maintaining multiple WM content dimensions.NEW & NOTEWORTHY Here, we found that working memory (WM)-driven ocular inflections-feature-specific pupillary and saccadic biases-were muted for memory items that were less behaviorally relevant. This work illustrates that functionally informative goal signals may extend as early as the sensorimotor periphery, that pupil size may be under more fine-grained control than originally thought, and that ocular signals carry multiple dimensions of cognitively relevant information.

Keywords: eye movements; pupillometry; visual attention; visual imagery; visual working memory.

Graphical abstract

Figure 1.

Task sequence and experimental conditions. An example sequence of events for a single-task trial in a high cue reliability block (left). Before each block began, participants were informed as to the likelihood that the retrocued working memory (WM) item would be tested (i.e., block-level cue reliability). In each trial, participants were shown two differently oriented Gabor patches to remember (detail in lower left corner). Then a directional cue (i.e., the retrocue) indicated which item would likely be tested. Following a memory delay, a probe cue specified the actual to-be-tested item. Subjects reported the remembered angle of orientation using a continuous recall wheel. Depending on the block reliability, the probe cue was more or less likely to be congruent with the cued item (i.e., valid trials), or incongruent (i.e., invalid trials). Tables depict the combination of block reliability and trial validity conditions in each experiment, and the theorized priority state for each WM item in those conditions.

Figure 2.

Schematic of eye-tracking data preprocessing procedures. Pupil velocity outliers were identified from the speed array, and were then rejected and interpolated. Identified blink events were then passed on to the gaze preprocessing pipeline (experiment 2).

Figure 3.

Behavioral results for both experiments. Signed response error distributions for each trial and block type in experiments 1 (A) and 2 (B). C and E: the mean response error for each trial and block type. D and F: the mean retrocue effect (i.e., difference score between valid and invalid trials) across the block reliability conditions. Points represent individual subject means, error bars represent 95% confidence interval (CI), and *individual data points that were cut off from the figure for illustrative purposes (but were included in the analysis).

Figure 4.

Task-evoked pupillary time-courses for all trial and block reliability conditions. A and B: trials when the darker item was cued are plotted in bolder traces, whereas trials when the brighter item was cued are plotted in lighter traces. Error bands represent ±1 SE. A: experiment 1 pupil traces for high reliable cue conditions (coral) and less reliable cue conditions (teal). B: experiment 2 pupil response for 95% (coral), 80% (teal), and 65% (purple) cue reliability conditions. C: experiment 1 grand average difference between dark and bright pupil traces, for each block condition. D: experiment 2 grand average differences between dark and bright pupil traces, for each block condition. The permutation test significance clusters within high reliability conditions are marked with coral shaded bars, and trend-level comparisons between block conditions are marked with gray shaded bars.

Figure 5.

Individual variation in pupillary working memory (WM) strength. A and C: distribution of vividness of visual imagery questionnaire (VVIQ) scores for experiments 1 and 2. B and D: the magnitude of each individual’s pupillary WM signal (dark − bright difference) as a function of self-reported visual imagery strength (VVIQ score). Each point corresponds to a subject’s epoched mean pupil size for dark − bright conditions in the period from 500 to 2,500 ms. E: pupil traces for subjects in the bottom, middle, and top 1/3 of VVIQ scores (labeled as weak, moderate, and strong imagers, respectively) for experiments 1 and 2.

Figure 6.

A: a heatmap of the horizontal and vertical gaze position difference for left- vs. right-cued trials, zoomed in to the fixation location. B: the horizontal gaze path following the directional retrocue, as a function of whether the left or right working memory (WM) item was cued. C: average frequency of microsaccade, classified into “toward” and “away” directions, across block reliability levels. D: individual subject mean frequency differences across block conditions over the 250–500 ms and 500–750 ms windows. **P < 0.01; ***P < 0.001. E: average horizontal coordinate for the identified gaze shift end points, for left/right cues, in the epoch of 1,000–2,500 ms after retrocue onset. For time series plots, error bands represent ±1 SEM, and shaded color bars at the bottom mark the time window of significant permutation test differences. For point estimates, error bars represent 95% confidence interval (CI).