The role of observers' gaze behaviour when watching object manipulation tasks: predicting and evaluating the consequences of action

Abstract

When watching an actor manipulate objects, observers, like the actor, naturally direct their gaze to each object as the hand approaches and typically maintain gaze on the object until the hand departs. Here, we probed the function of observers' eye movements, focusing on two possibilities: (i) that observers' gaze behaviour arises from processes involved in the prediction of the target object of the actor's reaching movement and (ii) that this gaze behaviour supports the evaluation of mechanical events that arise from interactions between the actor's hand and objects. Observers watched an actor reach for and lift one of two presented objects. The observers' task was either to predict the target object or judge its weight. Proactive gaze behaviour, similar to that seen in self-guided action-observation, was seen in the weight judgement task, which requires evaluating mechanical events associated with lifting, but not in the target prediction task. We submit that an important function of gaze behaviour in self-guided action observation is the evaluation of mechanical events associated with interactions between the hand and object. By comparing predicted and actual mechanical events, observers, like actors, can gain knowledge about the world, including information about objects they may subsequently act upon.

Keywords: action–observation; gaze behaviour; object manipulation.

Figure 1.

Experimental setting. (a) Observers viewed the actor from the side. In all trials, the actor first picked up and replaced the start block and then picked up and replaced either the near or far block before returning the hand to the start position shown. The black squares show the locations of the blocks. The position of the observer's gaze and the positions of the tips of the actor's index finger and thumb were recorded. (b) Device used to control the effective weights of the near and far blocks in the weight judgement task. A computer-controlled linear motor was used to position a pair of trolleys along parallel rotating rods attached, via a string running through pulleys, to hooks located in the centre of the blocks. A single rod, trolley and block are shown.

Figure 2.

Gaze behaviour in the target prediction task and corresponding self-guided action–observation condition. (a,b) Gaze and hand paths during periods of fixation and tracking (i.e. excluding saccades) in the plane of movement as viewed by the observer. Data from all trials by all participants are superimposed. For each trial, data are shown from the time at which the start block was replaced, just prior to the onset of the hand movement away from the start block, to the offset of the hand movement to the target block. Separate plots are shown for hand movements from the start block to the (i) near and (ii) far targets. Gaze paths are coloured red, blue or green depending on whether they are from saccadic, tracking or fixation trials, respectively, and hand paths are coloured grey. (c,d) Percentages of saccadic, tracking and fixation trials per participant in the action–observation (c) and target prediction (d) tasks.

Figure 3.

Gaze behaviour in the weight judgement task and corresponding self-guided action–observation condition. (a,b) Gaze and hand paths during periods of fixation and tracking (i.e. excluding saccades) in the plane of movement as viewed by the observer. Data from all trials by all participants are shown. For each trial, data are shown from the time at which the start block was replaced, just prior to the onset of the hand movement away from the start block, to the offset of the hand movement to the target block. Separate plots are shown for hand movements from the start block to the (i) near and (ii) far targets. Gaze paths are coloured red or green depending on whether they are from saccadic or fixation trials, respectively, and hand paths are coloured grey. (No tracking trials were observed.) Although the start block was fixated in almost all trials, some of the fixations do not appear in the figure because gaze shift away from the start block before the start block was replaced. (c,d) Vertical hand (grey) and gaze (red) positions, as a function of time, during the lift and replace of the target block in the weight judgement (c) and self-guided action observation (d) tasks. The continuous changes in vertical gaze position during fixation periods indicate that gaze tended to track the actor's lift and replace motion. Data from all saccadic trials in which the far block was the target, and weighed 3 N, are shown for a representative participant. Note that a larger number of such trials were collected in the weight judgement task than in the corresponding self-guided action–observation condition. For each trial, data are shown for the time period during which the actor's grip force exceeded 0.5 N, with an additional 50 ms before and after.