A functional model for studying common trends across trial time in eye tracking experiments

Abstract

Eye tracking (ET) experiments commonly record the continuous trajectory of a subject's gaze on a two-dimensional screen throughout repeated presentations of stimuli (referred to as trials). Even though the continuous path of gaze is recorded during each trial, commonly derived outcomes for analysis collapse the data into simple summaries, such as looking times in regions of interest, latency to looking at stimuli, number of stimuli viewed, number of fixations or fixation length. In order to retain information in trial time, we utilize functional data analysis (FDA) for the first time in literature in the analysis of ET data. More specifically, novel functional outcomes for ET data, referred to as viewing profiles, are introduced that capture the common gazing trends across trial time which are lost in traditional data summaries. Mean and variation of the proposed functional outcomes across subjects are then modeled using functional principal components analysis. Applications to data from a visual exploration paradigm conducted by the Autism Biomarkers Consortium for Clinical Trials showcase the novel insights gained from the proposed FDA approach, including significant group differences between children diagnosed with autism and their typically developing peers in their consistency of looking at faces early on in trial time.

Keywords: Autism spectrum disorder; Eye tracking; Functional data analysis; Functional principal components analysis; Multilevel functional principal component analysis.

Fig. 1:

(a) Derivation of ET outcomes in traditional analyses. Sample arrays from the visual exploration paradigm contained 5 full-color stimuli: face with direct gaze (social object, black bar inserted over eyes to protect identity of actress), outline of face filled with blurred pattern, bird, mobile phone, and motor vehicle. (b) Derivation of functional ET outcomes (viewing profiles) retaining information across trial time.

Fig. 2:

(a) A trial-level indicator for the initial gaze being at the social object where the initial gaze lasts approximately 400 ms. (b) A trial-level indicator for the initial gaze being at the social object where a child’s initial gaze is on a nonsocial object. (c) A sample viewing profile for a subject from baseline in our application to the VE paradigm. (d) The longitudinally observed (T1, T2, T3) viewing profiles of the same sample subject from our application to the VE paradigm.

Fig. 3:

(a) Estimated overall mean viewing profile, μ(t). (b) Estimated group level shifts from the overall mean, η_d(t), d = 1,2. (c) Estimated mean viewing profiles, μ(t) + η_d(t), d = 1,2, for the two diagnosis groups (ASD and TD).

Fig. 4:

(a),(d) The first two leading eigenfunctions, ${\varphi }_k^{(d)}(t)$, k = 1, 2, d = 1, 2, estimated from FPCA of the viewing profiles in the VE application. (b), (c) Group-specific mean estimates plus and minus two times the amount of variation along the leading eigenfunction, $\mu (t)+{\eta }_d(t)\pm 2\sqrt{{\lambda }_1^{(d)}}{\varphi }_1^{(d)}(t)$, d = 1, 2. (e), (f) Group-specific mean estimates plus and minus two times the amount of variation along the second leading eigenfunction, $\mu (t)+{\eta }_d(t)\pm 2\sqrt{{\lambda }_1^{(d)}}{\varphi }_2^{(d)}(t)$, d = 1, 2.

Fig. 5:

(a) Estimated first two leading eigenscores for the ASD group. Three ASD subjects with either the first or the second leading eigenscore outside 2 standard deviations from the mean are highlighted in green and red. (b) Estimated first two leading eigenscores for the TD group. (c) Viewing profiles of the three outlying ASD subjects. (Profiles for subjects highlighted by a red dot, circled dot and green dot are given in solid red, dashed red and solid green.)

Fig. 6:

(a) Estimated overall mean viewing profile, μ(t). (b) Estimated group-visit shifts from the overall mean, η_dj(t), d = 1, 2, j = 1, 2, 3. (c) Estimated mean viewing profiles, μ(t)+η_d·(t), d = 1, 2, for the two diagnosis groups (ASD and TD). (d) Estimated mean viewing profiles, μ(t)+η_·j(t), j = 1, 2, 3, at the three visits (T1, T2 and T3) in the VE application.

Fig. 7:

(a), (d), (g), (h) The first two leading subject- and visit-level eigenfunctions, ϕ_k(t), k = 1, 2 (a, d), ψ_k(t), k = 1, 2 (g, h) estimated from MFPCA. (b), (c), (e), (f) Group-specific mean estimates plus and minus two times the amount of variation along the leading two subject-level eigenfunctions, $\mu (t)+{\eta }_{d\cdot }(t)\pm 2\sqrt{{\lambda }_1}{\varphi }_1(t)$ (b, c), and $\mu (t)+{\eta }_{d\cdot }(t)\pm 2\sqrt{{\lambda }_2}{\varphi }_2(t)$ (e, f). (h), (i), (k), (l) Group-specific mean estimates plus and minus two times the amount of variation along the leading two visit-level eigenfunctions, $\mu (t)+{\eta }_{d\cdot }\pm 2\sqrt{{\nu }_1}{\psi }_1(t)$ (h, i), and $\mu (t)+{\eta }_{d\cdot }\pm 2\sqrt{{\nu }_2}{\psi }_2(t)$ (k. l).

Fig. 8:

(a) Estimated first two leading subject-level eigenscores for the ASD group. Three ASD subjects with either the first or the second leading eigenscore outside 2 standard deviations from the mean, also identified in the baseline analysis, are highlighted in green and red. (b) Estimated first two leading subject-level eigenscores for the TD group. (c), (d) Estimated first two leading visit-level eigenscores for the ASD and TD groups, respectively.

Fig. 9:

(a) Significant negative correlations are detected between average consistency of the initial gaze being at the social object and the average logarithm of latency to social object in the entire data (r = −.758, p-value< .001) as well as the ASD (r = −.768, p-value< .001) and TD groups (r = −.678, p-value< .001) (b), (c) Significant negative correlations are also detected between the leading subject-level eigenscores (positive scores signaling higher consistency) in both ASD and TD groups (ASD: r = −.0.706, p-value < .001, TD: r = −.678, p-value < .001).