Integrated Bayesian models of learning and decision making for saccadic eye movements

Abstract

The neurophysiology of eye movements has been studied extensively, and several computational models have been proposed for decision-making processes that underlie the generation of eye movements towards a visual stimulus in a situation of uncertainty. One class of models, known as linear rise-to-threshold models, provides an economical, yet broadly applicable, explanation for the observed variability in the latency between the onset of a peripheral visual target and the saccade towards it. So far, however, these models do not account for the dynamics of learning across a sequence of stimuli, and they do not apply to situations in which subjects are exposed to events with conditional probabilities. In this methodological paper, we extend the class of linear rise-to-threshold models to address these limitations. Specifically, we reformulate previous models in terms of a generative, hierarchical model, by combining two separate sub-models that account for the interplay between learning of target locations across trials and the decision-making process within trials. We derive a maximum-likelihood scheme for parameter estimation as well as model comparison on the basis of log likelihood ratios. The utility of the integrated model is demonstrated by applying it to empirical saccade data acquired from three healthy subjects. Model comparison is used (i) to show that eye movements do not only reflect marginal but also conditional probabilities of target locations, and (ii) to reveal subject-specific learning profiles over trials. These individual learning profiles are sufficiently distinct that test samples can be successfully mapped onto the correct subject by a naïve Bayes classifier. Altogether, our approach extends the class of linear rise-to-threshold models of saccadic decision making, overcomes some of their previous limitations, and enables statistical inference both about learning of target locations across trials and the decision-making process within trials.

Fig. 1

Experimental design. A complete session consists of 5 blocks, each of which contains 150 trials generated from the same block-specific transition matrix. All matrices shown in the main text were used to generate samples in each session. A trial consists of three consecutive stages: a fixation screen (showing a central red fixation dot); a target screen (showing both the fixation dot and an additional leftward or rightward target dot); an inter-trial interval (showing a black screen until the beginning of the next trial).

Fig. 2

Neuronal responses of a decision process and translation into a computational model. (a) Neuronal responses prior to a saccade from three trials. In their experiment, Schall and Thompson (1999) trained rhesus monkeys to stare at a central fixation stimulus and, as soon as eight secondary targets appeared, to elicit a saccade towards the oddball. The targets were arranged radially around the central fixation stimulus, and the location of the oddball was random. The diagram shows the recorded activity of single movement-related neurons in the saccadic movement maps of the frontal eye fields (FEF). Trials were grouped into those with slow, medium and fast saccades. The three plots show the averaged activity within these groups of trials, in each trial taking the activity from that neuronal response field corresponding to the correct target location. The activity patterns show that there is a fairly constant biophysical threshold at which a saccade is irrevocably elicited (grey bar) whereas the rate at which the signals rise varies between the groups of saccades. (Reprinted, with permission, from the Annual Review of Neuroscience, Volume 22 (c) 1999 by Annual Reviews, www.annualreviews.org) (b) Translation into a computational model of the decision process for a single trial. The rising activation in the FEFs is modelled as a linearly rising decision signal $S$. It starts off at an initial level $S_0$ and rises at a variable rate until reaching threshold $S_T$ at time $\tau$.

Fig. 3

Intra-trial model parameterization. The proposed intra-trial model has three free parameters, represented by grey arrows. (i) $\rho$ and (ii) $\sigma$ determine the mean and the standard deviation of the normally distributed slope of the decision signal that corresponds to the true target location of the current trial. The larger $\rho$, the shorter the predicted saccade latency $\tau$. The larger $\sigma$, the larger the variability of the distribution of $\tau$. (iii) $\vartheta$ specifies the threshold the decision signal has to reach in order to evoke a saccade. The larger $\vartheta$, the longer the latency and the less the influence of the initial value $S_0$.

Fig. 4

Log prior ratios predicted by the alternative inter-trial models. Each diagram is based on the combination of a particular block structure (transition-oriented, state-oriented, or uniform) and a particular inter-trial model (‘transition’ model, ‘state’ model, or ‘uniform’ model). For each trial, the diagrams show the target location (black dots), with high and low markers indicating leftward and rightward targets. Furthermore, they show the log ratio between the prior probability of the true and the false target location (grey squares) as well as a prediction for this log ratio, generated by the respective inter-trial model (black crosses). Since the models are always initialized with uniform priors, the predicted log ratio for trial $k=1$ is $ln(0.5/0.5)=0$. The upper left diagram, for example, shows how the ‘transition’ model gradually adapts to the transition-oriented block structure underlying the observed sequence of trials. By contrast, the central diagram in the left column shows that the ‘state’ model is incapable of learning the structure of a transition-oriented block.

Fig. 5

Histogram of latencies and reciprocals. The diagrams are based on 28 blocks containing 3 866 trials from subject S-1. While the latencies themselves have often been described as log normally distributed (Glimcher, 2003), their reciprocals can be approximated by a normal distribution (Carpenter & Williams, 1995).

Fig. 6

Saccade latencies and error bars. (a) Saccade latencies versus true state probabilities, based on all trials from state-oriented blocks across all subjects. (b) Saccade latencies versus true transition probabilities, based on all transition-oriented blocks across all subjects. Both diagrams show how saccade latencies decrease with increasing true probability of the respective target location.

Fig. 7

Averaged observed and predicted latencies. The diagram shows saccade latencies from an additional experimental session subject S-1 was engaged in. The dataset consists of 10 transition-oriented blocks, each designed to contain an identical left/right sequence of 150 target locations. The diagram shows the trial-by-trial target locations as separate lines of black dots at the bottom (leftward targets: lower line; rightward targets: upper line). The observed latencies, averaged over these 10 sessions, are plotted in black, and their respective model predictions in grey. Observations and predictions of trials in which the target location has stayed on the same side are shown as small black dots and grey circles, respectively. Observations and predictions of trials in which the target location has just switched to the other side are depicted as black crosses and grey squares, respectively. Predicted reaction times (grey circles and squares) show two key features of an ideal observer who is sensitive to transition probabilities. First, whenever there is a sequence of trials with identical target locations (a ‘run’), reaction times drop continuously as the estimated prior probability of that target location increases. Second, whenever the target changes to the other side (a ‘switch’), there is a single long-reaction-time trial, followed by a return to the previous, lower level of reaction times.

Fig. 8

Averaged observed versus predicted reciprocal latencies from Fig. 7. In each of the diagrams, predicted reciprocal latencies ($y$-axis) are plotted against their observations ($x$-axis). The diagrams are based on the ‘transition,’ the ‘state,’ and the ‘uniform’ model, respectively, applied to the same dataset as in Fig. 7. Thus, the $x$-coordinates of the data points are the same in all three diagrams, whereas their $y$-coordinates differ. Predictions were generated by the alternative models after being individually fitted to the data. The main diagonal represents a perfect match between observations and predictions.

Fig. 9

Negative log likelihood of the inter-trial models fitted to alternative subsets of the data. Each diagram shows the negative likelihood of the ‘uniform,’ the ‘state,’ and the ‘transition’ model when fitted to the data of a particular subject (rows) confronted with a sequence of target locations generated from a uniform, state-oriented, or transition-oriented block (columns). The rightmost column shows the negative likelihood of the model fitted to all blocks of a particular subject. The smaller the negative log likelihood, the better the model fit.

Box I