Serial dependence in visual perception: A review

Abstract

How does the visual system represent continuity in the constantly changing visual input? A recent proposal is that vision is serially dependent: Stimuli seen a moment ago influence what we perceive in the present. In line with this, recent frameworks suggest that the visual system anticipates whether an object seen at one moment is the same as the one seen a moment ago, binding visual representations across consecutive perceptual episodes. A growing body of work supports this view, revealing signatures of serial dependence in many diverse visual tasks. Yet, the variety of disparate findings and interpretations calls for a more general picture. Here, we survey the main paradigms and results over the past decade. We also focus on the challenge of finding a relationship between serial dependence and the concept of "object identity," taking centuries-long history of research into account. Among the seemingly contrasting findings on serial dependence, we highlight common patterns that may elucidate the nature of this phenomenon and attempt to identify questions that are unanswered.

Figure 1.

Serial dependence in visual perception and decision-making. (A) Our everyday environment is made of relatively stable and temporally correlated visual features: As we take a walk in the park, the objects around us (e.g., the leaves and trees) tend to remain the same, despite changes in luminance patterns and viewpoints. (B) To exploit such temporal continuity, the representation of a visual object (here illustrated as a probability distribution over stimulus space) can propagate from one moment to the next, biasing visual representations toward the recent past. This leads to systematic errors in perceptual decisions, which tend to be pushed toward the direction of the previous stimulus. (C–E) Three possible scenarios illustrating different accounts of the nature of representations involved in serial dependence; two different objects—the leaves and the tree—are shown inside green and brown circles, respectively. (C) Serial dependence can occur at the level of low-level visual features (e.g., orientation, motion, color) and independently of object-level representations. (D) Serial dependence can occur only for visual features of the same object. (E) Serial dependence can occur at more abstract levels of representation, where features of objects are extrapolated and reduced to elementary representations required by the task (e.g., both the tilt of the tree and of the leave can be represented as a tilted line). Note that, as in the “low-level” scenario, high-level serial dependence can be object independent. (The picture in A is from Parc de Milan, Lausanne, Switzerland.)

Figure 2.

Typical paradigms and findings in serial dependence research. (A) In a standard orientation adjustment task, an oriented stimulus (e.g., Gabor) is briefly presented and followed by a noise mask. After the mask, observers reproduce the perceived orientation of the Gabor by rotating a response tool (here a simple line). Serial dependence is seen in the adjustment errors (gray dots in the bottom plot) as a function of the difference in orientation between consecutive trials (Δ: previous minus present orientation). Typically, errors deviate toward the direction of the previous stimulus orientation (attractive bias, highlighted in the green regions of the plot), when Δ is small, following the shape of the first derivative of a Gaussian function (black curve). The dots in the bottom plots are from a simulated observer. (B) An example of a sequential dual task in which an adjustment task is followed by a forced-choice task. One of two stimuli (the inducer) is cued in the adjustment task and observers reproduce its orientation. In the “test” display, two other stimuli are presented, one at the same location as the inducer. In the forced-choice task, observers make a perceptual judgment (e.g., comparison or equality judgment) about the two test stimuli. Serial dependence is evident in the forced-choice task, as a shift in the perceived orientation of the stimulus at the inducer location, compared to the stimulus at the other “unbiased” location. The plot shows a simulated pattern of data resembling the findings of Fritsche and colleagues (2017), where the inducer caused repulsive biases: To be perceived as identical to the unbiased stimulus, the test stimulus at the inducer location had to be slightly tilted in the opposite direction of the inducer (positive Δs indicate when the unbiased stimulus was more clockwise). (C) The typical structure of a postcueing paradigm investigating serial dependence in motion direction between trials containing two clouds of moving dots. A postcue indicates the color of the cloud to report. The plot shows a simulated pattern of results based on the findings from Fischer and colleagues (2020): Serial dependence is influenced by the congruency between the feature cued on present and previous trials.

Figure 3.

Contrasting forces in serial dependence. (A) Serial dependence is typically positive when the difference between the previous and present stimulus is small and slightly negative when the difference is large (Δ, here indicating the difference in orientation between the previous and present stimulus, data are from 48 subjects performing an orientation adjustment task with low spatial frequency Gabors; Ceylan et al., 2021). (B) The coexistence of these two opposite biases can be modeled as the additive effect of a weighting function narrowly tuned toward the recent past (the green distribution) and one more broadly tuned away (the brown negative distribution). If observers weigh the previous stimulus according to the two distributions, the resulting pattern resembles A. (C) Interindividual variability in the dominance of the positive and negative components of serial dependence. Typically, the positive bias dominates; some subjects, however, show no bias or only negative serial dependence. (D) Two curves with a difference of Gaussian fit depicting the pattern for the top five observers showing positive serial dependence (green, corresponding to the green square in C) and the top five observers showing only negative biases (brown, corresponding to the brown square in C). The proportion of subjects showing positive serial dependence in this data set is 70% with an effect size of d′ = 0.63 (Cohen's d).