Gaze movements and spatial working memory in collision avoidance: a traffic intersection task

Abstract

Street crossing under traffic is an everyday activity including collision detection as well as avoidance of objects in the path of motion. Such tasks demand extraction and representation of spatio-temporal information about relevant obstacles in an optimized format. Relevant task information is extracted visually by the use of gaze movements and represented in spatial working memory. In a virtual reality traffic intersection task, subjects are confronted with a two-lane intersection where cars are appearing with different frequencies, corresponding to high and low traffic densities. Under free observation and exploration of the scenery (using unrestricted eye and head movements) the overall task for the subjects was to predict the potential-of-collision (POC) of the cars or to adjust an adequate driving speed in order to cross the intersection without collision (i.e., to find the free space for crossing). In a series of experiments, gaze movement parameters, task performance, and the representation of car positions within working memory at distinct time points were assessed in normal subjects as well as in neurological patients suffering from homonymous hemianopia. In the following, we review the findings of these experiments together with other studies and provide a new perspective of the role of gaze behavior and spatial memory in collision detection and avoidance, focusing on the following questions: (1) which sensory variables can be identified supporting adequate collision detection? (2) How do gaze movements and working memory contribute to collision avoidance when multiple moving objects are present and (3) how do they correlate with task performance? (4) How do patients with homonymous visual field defects (HVFDs) use gaze movements and working memory to compensate for visual field loss? In conclusion, we extend the theory of collision detection and avoidance in the case of multiple moving objects and provide a new perspective on the combined operation of external (bottom-up) and internal (top-down) cues in a traffic intersection task.

Keywords: collision avoidance; field loss compensation; gaze movements; homonymous hemianopia; potential-of-collision; spatial working memory; traffic intersection task; visual impairment.

Figure 1

Overview of the traffic intersection experiments used to analyze the function of gaze and visuo-spatial memory in the task of collision prediction or avoidance in normal and visually impaired subjects. The overall structure of the intersection task used in all our experiments was as follows: beginning from a start point, subjects approached (passively or interactively, i.e., controlling their speed) the intersection while visually observing of the traffic on the intersecting street in order to predict or avoid a collision. The cars on the intersecting street had different colors and were uniformly distributed over two lanes (right-hand traffic); their number could be varied to allow for different traffic densities. The speed of the traffic cars was always constant with 50 km/h and their travel paths started and ended in tunnels. In passive trials, subjects approached until a decision point where estimates about a collision or the positions of traffic cars were made. In interactive trials, subjects were allowed to adjust their own driving speed (within certain limits) within the approach section between the start and end points marked in the figure. From there on, movement was extrapolated with constant speed and the occurrence of collisions was recorded.

Figure 2

Large-field projection screen (presenting the approach phase of the experiment) and seat used in the experiments. The screen provides a large field of view of 150 by 70° in a seated but otherwise unrestricted subject. Inset: eye and head tracking devices with 60Hz sampling frequency (head: ARTtrack/DTrack from A.R.T., eye: model 501 from Applied Science Laboratories).

Figure 3

Collision detection performance. (A) Distribution of individual hit and false alarm rates shown for all 25 subjects (red squares). Data points significantly different from zero are marked by open symbols. Values for the mean and standard deviation of the hit (value: 59 ± 20%) and the false alarm rates (value: 30 ± 14%) are depicted as a red cross. The red circle denotes that subject (#36) for which scan-path details are shown in Figure 4. (B) Box plots of the averaged discriminability index (d-prime; left) and the averaged criterion for an observer bias (C; right). Statistical effects (one-sample T-Test, test value: zero) are presented for criterion C and d-prime (^*p < 0.05; ^***p < 0.001).

Figure 4

Scan-path examples of a representative subject (#36; hit rate: 55%, false-alarm rate: 22.5%; see Figure 3A). (A) Gaze scan-paths for all 80 trials performed by the subject are plotted together. Blue parts of the scan-paths denote fixations and red ones saccades. The left-right-left pattern of gaze movement combined with local scanning was found as general pattern in all subjects. Note the inward slant of the blue segments, indicating that object fixations almost exclusively landed on to cars approaching the intersection (d-prime and observer bias are reported for this subject). (B) Exemplary scan-path of the approach to the intersection for a single hit trial. The positions of all cars over time are shown in blue; the collision relevant car is depicted in red, i.e., the red line hits the subjects' trajectory (vertical midline) at time 11.7 s. Position lines of all cars after the time the cars disappear are drawn with transparency. Fixational elements of the gaze scan-path are drawn as black, positions during saccades as white circles. Differences in time-to-collision for all cars with relevant POC are given relative to the colliding one (numeric values in green). (C) Another exemplary scan-path of the approach to the intersection for a hit trial where no relevant fixations to the colliding car occur. (D) Exemplary scan-path of the approach to the intersection for a single miss trial. Please note, despite fixating the collision relevant car for about 2 s, the subject failed to detect the high POC of that car.

Figure 5

Schematic account of the allocation of overt attention (i.e., distribution of fixations) during the approach to the intersection is scaled by the subject's distance to the intersection (blue: furthest, green: intermediate, and orange: closest) combined with the position of the traffic cars. Additionally, only cars moving toward the intersection are considered (arrows).

Figure 6

Distribution of task relevant fixations applied in all crash trials shown as proportion of fixations on cars (during the approach) for good performers (subjects with hit rates above 50%, red bars) and for poor ones (subjects with miss rates above 50%, gray bars). Redrawn from Roth (2011).

Figure 7

Task specific representation of cars in working memory shaped by their POC (gray: cars moving from left to right; red: cars moving from right to left). (A) Distribution of number of cars positioned on the intersection in the memory recall phase (i.e., reconstructing the traffic scene) of the traffic intersection task [redrawn from Hardiess and Mallot (2010)]. (B) Frequency of correct detection of change due to cars removed from all positions on the intersecting street (i.e., only hit trials are considered) in the change detection version of the traffic intersection task [redrawn from Hardiess et al. (2011)]. The smooth curves show normal distributions fitted to the data.

Figure 8

Visualization of gaze trajectory. Each plot demonstrates the participant's gaze pattern during approach to the intersection for a representative trial with 50% traffic density. A normal subject (A), an adequate-HVFD_A patient with right hemianopia (B), and an inadequate-HVFD_I patient with right hemianopia (C) are shown. Gaze position is depicted in black (fixations shown as circles and saccades as lines). Gray transparent areas are beyond the visual field boundaries. The blue lines represent the courses of the traffic cars and red segments indicate when vehicles are moving on collision course. Speed adjustments of the participant (acceleration or deceleration) result in kinks on the blue lines. The HVFD_A patient shows more gaze shifts, more fixations on cars, larger saccades, larger mean gaze eccentricity, and more speed adjustments (kinks) than the HVFD_I patient, who demonstrates decreased gaze activity. Trial duration is similar between the normal subject and the HVFD_A patient, while the HVFD_I patient completes trials in a shorter period of time. Redrawn from Papageorgiou et al. (2012c).