Adaptation to the Direction of Others' Gaze: A Review

Abstract

The direction of another person's gaze provides us with a strong cue to their intentions and future actions, and, correspondingly, the human visual system has evolved to extract information about others' gaze from the sensory stream. The perception of gaze is a remarkably plastic process: adaptation to a particular direction of gaze over a matter of seconds or minutes can cause marked aftereffects in our sense of where other people are looking. In this review, we first discuss the measurement, specificity, and neural correlates of gaze aftereffects. We then examine how studies that have explored the perceptual and neural determinants of gaze aftereffects have provided key insights into the nature of how other people's gaze direction is represented within the visual hierarchy. This includes the level of perceptual representation of gaze direction (e.g., relating to integrated vs. local facial features) and the interaction of this system with higher-level social-cognitive functions, such as theory of mind. Moreover, computational modeling of data from behavioral studies of gaze adaptation allows us to make inferences about the functional principles that govern the neural encoding of gaze direction. This in turn provides a foundation for testing computational theories of neuropsychiatric conditions in which gaze processing is compromised, such as autism.

Keywords: face perception; gaze direction; sensory coding; social attention; visual adaptation.

FIGURE 1

Illustration of the magnitude of gaze aftereffects based on fitted data from 28 neurotypical adults (Palmer et al., 2018). Leftmost column denotes the adapting condition. Subsequent columns represent the perceived direction of gaze of faces with eyes averted horizontally by –10, 0, and +10 degrees, respectively.

FIGURE 2

The phenomenon of gaze adaptation demonstrates that eye direction detection is a remarkably plastic process modifiable by the recent diet of stimulation. Eye direction information feeds into theory of mind by providing a cue about other people’s focus of attention, their knowledge, and their intentions (Baron-Cohen, 1995). The work of Teufel et al. (2009, 2013) indicates further that theory of mind is itself able to modulate the strength of adaptation A within the eye direction detector.

FIGURE 3

Contrasting frames of reference. (A) In this example, the individual on the right has gaze averted 90° relative to the observer (the ‘first person’ reference frame). In contrast, their eye direction relative to their own head is 45° (the ‘second person’ reference frame). (B) Across these face images, the direction of gaze is in the same rightwards direction relative to the viewer, but signaled by different combinations of head and eye angle. There is evidence that the brain contains representations of where other people are looking relative to oneself that are engaged regardless of the particular head and eye direction that combine to signal this direction of gaze in a given image (Palmer and Clifford, 2017b; Clifford, 2018; discussed in Section “Specificity of Gaze Adaptation”).

FIGURE 4

Inside the Eye Direction Detector. Schematic representation of the functional architecture proposed by Palmer and Clifford (2017a) to underlie the coding of horizontal gaze direction. Gaze direction is encoded by the pattern of activation across three channels tuned to leftward, direct, and rightward, respectively. The outputs of these channels are then combined through a process of divisive normalization of an opponent left-right signal to generate a metric estimate of gaze direction.

FIGURE 5

(A) The simulated effect of normalization mechanisms on the tuning of perceptual aftereffects. This figure shows the size of perceptual aftereffects predicted by the model of perceived gaze direction (illustrated in Figure 4) following adaptation to 25° leftward gaze, across a range of test stimulus gaze directions. In the model of perceived gaze direction, the encoded gaze direction is normalized to the summed activation across gaze-selective sensory channels. The plotted lines show the simulated aftereffects for a series of models ranging from ‘full’ normalization of the encoded gaze direction to a complete lack of normalization. As the degree of normalization is reduced, the tuning of aftereffects across test gaze directions change in a systematic way. Thus, perceptual aftereffects observed following adaptation to gaze direction may be indicative of differences between individuals or groups in the operation of normalization mechanisms in the coding of gaze direction. (B) The simulated effects of channel adaptability on perceptual aftereffects. The plotted lines show the simulated aftereffects for a series of models with the same degree of normalization, but where exposure to the adapting stimulus results in either a stronger or weaker change in subsequent channel sensitivities. The degree of channel adaptability scales the magnitude of perceptual aftereffects, but has a less distinct effect on the tuning of aftereffects across stimulus gaze directions compared to the effect of varying normalization shown in (A).