Measuring inter- and intra-individual differences in visual scan patterns in a driving simulator experiment using active information storage

Abstract

Scan pattern analysis has been discussed as a promising tool in the context of real-time gaze-based applications. In particular, information-theoretic measures of scan path predictability, such as the gaze transition entropy (GTE), have been proposed for detecting relevant changes in user state or task demand. These measures model scan patterns as first-order Markov chains, assuming that only the location of the previous fixation is predictive of the next fixation in time. However, this assumption may not be sufficient in general, as recent research has shown that scan patterns may also exhibit more long-range temporal correlations. Thus, we here evaluate the active information storage (AIS) as a novel information-theoretic approach to quantifying scan path predictability in a dynamic task. In contrast to the GTE, the AIS provides means to statistically test and account for temporal correlations in scan path data beyond the previous last fixation. We compare AIS to GTE in a driving simulator experiment, in which participants drove in a highway scenario, where trials were defined based on an experimental manipulation that encouraged the driver to start an overtaking maneuver. Two levels of difficulty were realized by varying the time left to complete the task. We found that individual observers indeed showed temporal correlations beyond a single past fixation and that the length of the correlation varied between observers. No effect of task difficulty was observed on scan path predictability for either AIS or GTE, but we found a significant increase in predictability during overtaking. Importantly, for participants for which the first-order Markov chain assumption did not hold, this was only shown using AIS but not GTE. We conclude that accounting for longer time horizons in scan paths in a personalized fashion is beneficial for interpreting gaze pattern in dynamic tasks.

Fig 1. Definition of areas of interest (AOI) and experimental procedure.

(A) AOI defined based on their semantic meaning in the driving task. (B) Experimental procedure: Each trial comprises a baseline period of five fixations and the task interval that starts with the deceleration of the car in front of the ego-car (grey). The task is split into the period before lane change, between trial onset and changing onto the left lane (blue arrow and marker), and the period after lane change, between changing onto the left lane and changing back onto the right lane (red arrow and marker).

Fig 2. Schematic illustration of AIS estimation and relationship with other information-theoretic measures.

(A) Schematic illustration of relationship between AIS (blue box), conditional entropy of the next fixation given the past state (orange box) and the joint entropy between past state and next fixation (grey box). For ${\mathbf{X}}_{t-1}^{-}=X_{t-1}$, the conditional entropy is equivalent to the GTE. (Adapted from [38]). (B) Schematic illustration of the non-uniform embedding (modified from [14]). The blue box indicates all past variables up to a maximum lag, l_max, that are considered during the optimization of the past state for AIS estimation. The red marker indicates the next fixation, X_t, and blue markers indicate the variables selected by the optimization that comprise the optimized past state, ${\mathbf{X}}_{t-1}^{-}$.

Fig 3. Proportion of fixations and mean fixation duration.

(A) Relative fixation proportions for each AOI across all participants in the three different trial periods (LC = lane change). (B) Distribution of mean fixation duration across participants for the three trial periods and two levels of difficulty including (whiskers indicate 95% confidence intervals). (C) Mean fixation duration per participant for each trial period (error bars indicate 95% confidence intervals). (D) Mean fixation duration per participant for both difficulty levels (error bars indicate 95% confidence intervals).

Fig 4. Selected past variables per participant.

Lags, l, of past variables used for AIS estimation as identified by the estimation algorithm for each participant.

Fig 5. Mean values of LGTE (orange), LAIS (blue), and joint entropy (gray) as a function of trial period.

Mean per trial period for participants with lag l > 1 for (A) LGTE, (B) LAIS, (C) joint entropy between next fixation and past fixation at t − 1 (fixations used for LGTE estimation), (D) joint entropy between next fixation and past fixation at t − l (fixations used for LAIS estimation). Mean per trial period for participants with lag l = 1 for (E) LGTE, (F) LAIS, (G) joint entropy between next fixation and past fixation at t − 1 (fixations used for LGTE estimation), (H) joint entropy between next fixation and past fixation at t − l (fixations used for LAIS estimation). Error bars indicate the standard error of the mean.

Fig 6. Predicted values of mean LGTE (orange) and mean LAIS (blue) by linear mixed-effects model (LMEM).

Error bars indicate confidence intervals. (A) Mean LGTE values normalized by mean joint entropy for l > 1, (B) mean LAIS values normalized by mean joint entropy for l > 1, (C) mean LGTE values normalized by mean joint entropy for l = 1, (D) mean LAIS values normalized by mean joint entropy for l = 1.