Selective scanpath repetition during memory-guided visual search

Abstract

Visual search efficiency improves with repetition of a search display, yet the mechanisms behind these processing gains remain unclear. According to Scanpath Theory, memory retrieval is mediated by repetition of the pattern of eye movements or "scanpath" elicited during stimulus encoding. Using this framework, we tested the prediction that scanpath recapitulation reflects relational memory guidance during repeated search events. Younger and older subjects were instructed to find changing targets within flickering naturalistic scenes. Search efficiency (search time, number of fixations, fixation duration) and scanpath similarity (repetition) were compared across age groups for novel (V1) and repeated (V2) search events. Younger adults outperformed older adults on all efficiency measures at both V1 and V2, while the search time benefit for repeated viewing (V1-V2) did not differ by age. Fixation-binned scanpath similarity analyses revealed repetition of initial and final (but not middle) V1 fixations at V2, with older adults repeating more initial V1 fixations than young adults. In young adults only, early scanpath similarity correlated negatively with search time at test, indicating increased efficiency, whereas the similarity of V2 fixations to middle V1 fixations predicted poor search performance. We conclude that scanpath compression mediates increased search efficiency by selectively recapitulating encoding fixations that provide goal-relevant input. Extending Scanpath Theory, results suggest that scanpath repetition varies as a function of time and memory integrity.

Keywords: Eyetracking; relational memory; scanpath; visual search.

Figure 1

.Example stimulus image. (A) Unmanipulated image, (A’) Manipulated image where target is removed.

Figure 2.

A trial sequence from the modified flicker change detection/visual search task.

Figure 3.

Across-subject fixations for Trial 1, clustered according to the described algorithm. Clusters are indicated by variations in shape and colour. (A) Scene image to which fixation clusters correspond, (B) Unclustered fixations from all subjects, (C) Clusters given centre of δ > 50.

Figure 4.

Process for calculating controlled scanpath similarity. Characters are assigned to novel, repeated, and control fixations based on cluster locations. String-edit distance scores are computed for novel and repeated viewing strings and for control and repeated viewing strings. SEDc is an average of the string-edit distance scores for the repeated-viewing string to all 50 control strings. The controlled scanpath similarity score reflects the similarity of the repeated-viewing scanpath to its corresponding novel-viewing scanpath, controlled for saliency. The controlled scanpath similarity score has a baseline of 0.

Figure 5.

Visualization of sliding window similarity analysis applied to fixations from corresponding novel and repeated viewing scanpaths. Similarity is computed for all novel viewing fixations to: (A) all repeated viewing fixations, (B) the first three repeated viewing fixations, (C) the last three repeated viewing fixations. Panel D demonstrates how SED would be calculated for the comparisons depicted in panels A–C. SED scores (column 2, rows 3–5) reflect the minimum number of editing operations required to equate the 3-fixation sequences in the corresponding windows (indicated by the column and row numbers). For example, the editing cost of equating novel viewing window 3 (C-J-M) to repeated viewing window 3 (E-J-M) is 1, reflecting the single replacement operation required. The bottom row contains the average of SED scores at each novel viewing window corresponding to the analysis depicted in panel A. The highlighted rows contain the SED scores corresponding to the analyses depicted in panel B and panel C, respectively. Note, that the scores here reflect only the comparison of novel and repeated viewing strings (SED_i). In the actual analysis, these similarity scores are controlled for saliency (see Figure 4 for this process).

Figure 6.

Eye movement measures by age and repetition set. (A) mean search time (s), (B) mean number of fixations, (C) mean fixation duration (s). Error bars: +/− 1 SE.

Figure 7.

95% and 99% confidence intervals for controlled scanpath similarity for: (A) all 3-fixation repeated viewing windows across all 3-fixation novel viewing windows, averaged at each novel viewing window, (B) First three repeated viewing fixations across a 3-fixation sliding window of novel viewing fixations, (C) Last three repeated viewing fixations across a 3-fixation sliding window of novel viewing fixations. The 0 line marks baseline or chance similarity, where repeated viewing fixations are equally distant from novel viewing and saliency-based, control-generated fixations. Window lengths: OA = 38; YA = 24.

Figure 8.

Group mean correlation values for controlled scanpath similarity, averaged at the beginning, middle, and end of the scanpath, and repeated viewing search time. Correlation values are averaged within each age group. To generate confidence intervals, a distribution of correlation values was created for each subject (by sampling similarity and search scores) and for each age group (by sampling from the subject distributions).