Right place, right time: Spatiotemporal predictions guide attention in dynamic visual search

Abstract

Visual search is a fundamental element of human behavior and is predominantly studied in a laboratory setting using static displays. However, real-life search is often an extended process taking place in dynamic environments. We have designed a dynamic-search task in order to incorporate the temporal dimension into visual search. Using this task, we tested how participants learn and utilize spatiotemporal regularities embedded within the environment to guide performance. Participants searched for eight instances of a target that faded in and out of a display containing similarly transient distractors. In each trial, four of the eight targets appeared in a temporally predictable fashion with one target appearing in each of four spatially separated quadrants. The other four targets were spatially and temporally unpredictable. Participants' performance was significantly better for spatiotemporally predictable compared to unpredictable targets (Experiments 1-4). The effects were reliable over different patterns of spatiotemporal predictability (Experiment 2) and primarily reflected long-term learning over trials (Experiments 3, 4), although single-trial priming effects also contributed (Experiment 4). Eye-movement recordings (Experiment 1) revealed that spatiotemporal regularities guide attention proactively and dynamically. Taken together, our results show that regularities across both space and time can guide visual search and this guidance can primarily be attributed to robust long-term representations of these regularities. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Figure 1. Trial Schematic for Experiment 1

Note. (a) On each trial, participants searched for eight vertical bars among distractors—a yellow (gray) circle indicates the first predictable (unpredictable) target. Targets and distractors appeared and disappeared over the course of the trial. (b) The time course of a trial is depicted with the onset of trial events represented as rectangles (colored rectangles represent predictable targets while gray rectangles represent unpredictable targets. (c) Here is an example of a single participant’s target events across one block of trials. Predictable targets are represented as filled circles whereas unpredictable targets are represented as unfilled circles. The time within a trial is represented from yellow (light gray) to dark red (dark gray). Here it can be seen that predictable targets always appear at the same time and within the same quadrant (although the exact location within the quadrant varied). Unpredictable targets had variable onsets and could appear anywhere. See the online article for the color version of this figure.

Figure 2. Spatiotemporal Regularities in Dynamic Visual Search Guide Behavior

Note. (a) Mean accuracy for predictable and unpredictable targets. (b) Mean accuracy across the trial. Colors follow the convention in Figure 1. (c) Mean RT for predictable and unpredictable targets. (d) Mean RT across the trial. Error bars in bar graphs represent the standard error of the mean, individual participants are represented as light gray lines, and stars indicate a significant difference with a p-value < .05 in this and all subsequent figures. See the online article for the color version of this figure.

Figure 3. Gaze is Biased Towards Predictable Targets Earlier

Note. (a) The probability of a fixation landing in the “target” quadrant for a 5-second epoch around the “onset” of the target (Time 0). The target dynamics are indicated on the x axis and the moment fade in begins (0s), the moment max opacity is reached (2 s), and when the fade out begins (2.8 s) are labeled as well. Note the shading and color of this line is illustrative and does not actually reflect the color values of the target. Given there are four quadrants, the chance of being in any one quadrant would be .25. The onset indicates the moment that the targets were no longer completely transparent, although they were not necessarily visible at this moment and shaded areas around the line represent the 95% confidence interval. The probability of fixating a target quadrant was significantly higher for predictable targets early in the epoch and significantly lower during the later stage of the epoch. Significant time windows are marked with a solid red (black) line. (b) This pattern repeats throughout the different instances of the target. See the online article for the color version of this figure.

Figure 4. Increased and Faster Detection of Predictable Targets With Asynchronous Regularities

Note. (a) The trial schematic for Experiment 2. Each trial was divided into six time bins, and each observer was randomly assigned to four bins—a yellow (gray) circle indicates the first predictable (unpredictable) target. (b) Mean accuracy is plotted for predictable and unpredictable targets—individual participants are represented as light gray lines. (c) Mean accuracy is plotted across the trial split by predictability. Mean RT for predictable and unpredictable targets (d) averaged over the trial and (e) separately for target across the trial. Asterisks indicate a p-value < .05. See the online article for the color version of this figure.

Figure 5. Spatiotemporal Guidance Is Held in Long-Term Memory

Note. (a) The trial schematic for Experiment 3. In 40% of trials, the locations and timings of all targets were completely unpredictable. (b) Hypothetical results are shown for two scenarios: the predictability effect is dependent on single-trial priming (left) or the predictability effect is resistant to interference (right). (c) Mean accuracy is plotted for predictable and unpredictable targets separated by the previous trial type—individual participants are represented as light gray lines. Participant means for the average accuracy in fully random trials are also shown. (d) Accuracy in standard trials is plotted across the trial. (e) Mean RTs are plotted for predictable and unpredictable targets when the target was preceded by a standard trial, a random trial, as well as for when the trial itself was fully random. (f) The reaction times in standard trials across the entire trial. Asterisks indicate a p-value < .05. See the online article for the color version of this figure.

Figure 6. Trial-Wise Priming Contributes to Behavioral Benefits

Note. (a) Depiction of a trial schematic for Experiment 4. Forty percent of trials were repeated such that the timings and quadrants of the unpredictable targets were the same as in the previous trial. (b) Depicted are hypothetical results supporting single-trial priming effects (left) and without evidence for single-trial priming effects (right). It is of note that if we find evidence to support the notion of single trial priming this should manifest in better performance in the repeat vs. nonrepeat trials. (c) Accuracy measures for predictable targets (in all trials) and the unpredictable targets in the nonrepeat vs. repeat trials. (d) Here we see the effect of repeating a single trial over the course of a trial; (e) compares reaction times for predictable targets, nonrepeat trials, and repeat trials. In (f) we see this effect across a trial. Asterisks indicate a p-value < .05. See the online article for the color version of this figure.