Flipping the world upside down: Using eye tracking in virtual reality to study visual search in inverted scenes

Abstract

Image inversion is a powerful tool for investigating cognitive mechanisms of visual perception. However, studies have mainly used inversion in paradigms presented on twodimensional computer screens. It remains open whether disruptive effects of inversion also hold true in more naturalistic scenarios. In our study, we used scene inversion in virtual reality in combination with eye tracking to investigate the mechanisms of repeated visual search through three-dimensional immersive indoor scenes. Scene inversion affected all gaze and head measures except fixation durations and saccade amplitudes. Our behavioral results, surprisingly, did not entirely follow as hypothesized: While search efficiency dropped significantly in inverted scenes, participants did not utilize more memory as measured by search time slopes. This indicates that despite the disruption, participants did not try to compensate the increased difficulty by using more memory. Our study highlights the importance of investigating classical experimental paradigms in more naturalistic scenarios to advance research on daily human behavior.

Keywords: Eye movements; eye tracking; incidental memory; scene inversion; scene perception; virtual reality; visual search.

Figure 1.

Bird’s-eye view of one of the bathrooms and one sample view of all the scenes that were used in the experiment. Blue squares indicate the starting position of the participants and were not visible during searching.

(1)

(2)

(3)

(4)

Figure 2.

Response times of correct searches from trial 1 to 10 within one scene. Solid straight lines represent regression lines. Solid points indicate means calculated on log-transformed response times which were converted back to their original form for visual purposes, error bars indicate within-subject standard errors.

Figure 3.

Response time split up into initiation time (a), scanning time (b), and verification time (c). Solid points indicate means, error bars indicate within-subject standard errors, colored points represent individual participants. ***p < .001.

Figure 4.

Fixation measures. (a) Fixation durations. (b) Fixation count. (c) Fixated objects count. (d) Refixations. Solid points indicate means, error bars indicate within-subject standard errors, colored points represent individual participants. ***p < .001.

Figure 5.

Joint distributions of relative (a) and absolute (b) directions of gaze and head movements and their amplitudes as a function of scene orientation. The lower right plot in each (a) and (b) includes labels for the direction. The lighter the color, the more gaze or head movements were executed into the direction. Radial ticks represent degrees, while ticks from inside out represent saccade amplitudes.

Figure 6.

Mean proportion of absolute (a) gaze and (b) head directions as a function of scene orientation. Error bars indicate standard errors.

Figure 7.

Gaze latitude (a), and amplitudes for (b) gaze and (c) head movements. Solid points indicate means, error bars indicate withinsubject standard errors, colored points represent individual participants. In (a), the dashed red line represents the horizon. *p < .05, ***p < .001.