Why are you looking at me? It's because I'm talking, but mostly because I'm staring or not doing much

Abstract

Our attention is particularly driven toward faces, especially the eyes, and there is much debate over the factors that modulate this social attentional orienting. Most of the previous research has presented faces in isolation, and we tried to address this shortcoming by measuring people's eye movements whilst they observe more naturalistic and varied social interactions. Participants' eye movements were monitored whilst they watched three different types of social interactions (monologue, manual activity, active attentional misdirection), which were either accompanied by the corresponding audio as speech or by silence. Our results showed that (1) participants spent more time looking at the face when the person was giving a monologue, than when he/she was carrying out manual activities, and in the latter case they spent more time fixating on the person's hands. (2) Hearing speech significantly increases the amount of time participants spent looking at the face (this effect was relatively small), although this was not accounted for by any increase in mouth-oriented gaze. (3) Participants spent significantly more time fixating on the face when direct eye contact was established, and this drive to establish eye contact was significantly stronger in the manual activities than during the monologue. These results highlight people's strategic top-down control over when they attend to faces and the eyes, and support the view that we use our eyes to signal non-verbal information.

Keywords: Attention; Audio visual interaction; Direct eye gaze; Eye movements; Social attention; Social cognition; Speech perception.

Fig. 1

Stills of the Misdirection, Manual action and Monologue videos (left to right) with interest areas outlined. The face interest area consisted of an oval shape that covered the entire face. Two sub-interest areas were created which covered the eyes and the mouth. The hands interest area covered both of the actor’s hands, as well as the objects that were touched in each particular frame, and the size of the interest area was expanded so that it covered these objects

Fig. 2

Mean percent dwell times for fixating the face and hands as a function of video and sound condition (error bars denote ±1 standard error)

Fig. 3

Mean percent dwell times for fixating the eyes and mouth as a function of activity and sound condition (error bars denote ±1 standard error). These interest areas are subsets of the earlier analysed face interest area

Fig. 4

Mean percent dwell times for fixating the face as a function of whether the filmed person’s gaze was directed towards the observer (direct) or averted