Trial-history biases in evidence accumulation can give rise to apparent lapses

Abstract

Trial history biases and lapses are two of the most common suboptimalities observed during perceptual decision-making. These suboptimalities are routinely assumed to arise from distinct processes. However, several hints in the literature suggest that they covary in their prevalence and that their proposed neural substrates overlap - what could underlie these links? Here we demonstrate that history biases and apparent lapses can both arise from a common cognitive process that is normative under misbeliefs about non-stationarity in the world. This corresponds to an accumulation-to-bound model with history-dependent updates to the initial state of the accumulator. We test our model's predictions about the relative prevalence of history biases and lapses, and show that they are robustly borne out in two distinct rat decision-making datasets, including data from a novel reaction time task. Our model improves the ability to precisely predict decision-making dynamics within and across trials, by positing a process through which agents can generate quasi-stochastic choices.

Figure 1:. Trial history-dependent initial states give rise to apparent lapses

(A) Schematic of two commonly observed suboptimalities in decision-making: history biases (top) and lapses (bottom). (Left): Rat performing a perceptual decision making task, where it has to make one of two decisions (left, right) based on accumulated sensory evidence (auditory clicks on either side). (Top left): History biases i.e. a tendency for the decision on the current trial (n) to be inappropriately influenced by what happened on the previous trial (n−1) in addition to the accumulated sensory evidence. In this example, a previously rewarding leftward decision is likely to be repeated. (Top right): Typically assumed effect of history bias on the psychometric curve, which is the proportion of rightward decisions as a function of the stimulus strength. History biases are thought to most strongly affect decisions when the sensory evidence is weak i.e. around the inflection point of the curve (threshold parameter), shifting it horizontally. (Bottom left): Lapses i.e. a tendency to make seemingly random choices on some trials, irrespective of the accumulated sensory evidence. (Bottom right): Typically assumed effect of lapses on the psychometric curve is vertically scaling the endpoints or asymptotes of the curve. (B) Standard normative model of within-trial processing during evidence accumulation. (Top) Decision rule that produces the most accurate decisions in the shortest amount of time, in which a decision is made when the summed log-ratios of category priors and likelihoods exceeds one of two decision bounds. This corresponds to a drift-diffusion process where the prior term sets the initial state (I) and the rate of accumulating evidence sets the drift rate (μ). (Bottom left): Schematic of the generative model, where one of two hypotheses (H1, H2) produce noisy samples of evidence over time (ϵ_t). (Bottom right): Schematic of the aforementioned drift-diffusion process, showing a sample trajectory based on noisy evidence (bold line) that leads to a rightward decision when the positive bound is hit. Thin lines depict alternate trajectories based on different noisy instantiations of the same drift rate (black arrow). (C) Model of across trial processing that can accommodate several forms of prior updates. Past choices and outcomes can additively affect the initial state with different magnitudes (η) and exponentially decaying timescales (β) depending on whether they were wins (top left) or losses (top right). (Bottom): Example sequence of trials, labelled by whether they follow a previous win (triangles) or previous loss (circles) on right (R) or left (L) choices, showing the cumulative effect of trial history on initial state updates. Colors denote different initial state biases, same as (C). (D) Effect of initial state values on psychometric functions. Colors denote different initial state levels, towards the positive (blue) or negative (pink) decision bounds. Small deviations from 0 in the initial state (grey) lead to largely additive, horizontal biases in the psychometric curve whereas larger deviations (saturated colors) have more complex effects, additionally reducing its effective slope (dotted black lines) or “sensitivity” to the stimulus. (E) Pooling different initial state biases gives rise to apparent lapses. Psychometric function (black) pooled across trials with different initial state biases (due to history-based updating) has apparent lapses (purple arrow), moreover conditioning the psychometric curve on whether the previous trial was a rightward (blue) or leftward (pink) win reveals a modulation of these apparent lapses by trial history.

Figure 2:. History-dependent threshold and lapse rate modulations in a large-scale rat dataset

(A) Schematic of evidence accumulation task in rats: (Top): Phases of the ‘Poisson clicks’ task, including trial initiation in center port (left), evidence accumulation based on two streams of Poisson-distributed auditory clicks (middle) and choice report in one of two side ports followed by water reward for correct choices (right). (Bottom): Time-course of trial events in a typical trial. (B) Individual differences in history-dependence: Psychometric functions of three example rats from a large-scale dataset, displaying different kinds of history modulation. Choices are plotted conditioned on previous left (blue), right (pink) or all wins (black). (Left): Example rat with no history-dependence in choices, resembling the ideal observer. (Middle): Example rat with modulations of the threshold parameter alone, resembling the dominant conceptualization of history bias. (Right): Example rat with history-dependent modulation of both threshold and lapse rate parameter, similar to the majority of the population. (C) Dataset displays significant modulations of both threshold and lapse rate parameters: Scatters showing parameters of psychometric functions following leftward wins (post left, blue) or rightward wins (post right, pink). Each pair of connected gray points represents an individual animal, solid colored dots represent average parameter values across animals. Trial history does not significantly affect the sensitivity parameter (top left) but significantly affects left, right lapse rate and threshold parameters (top right and bottom panels). (D) Scatter comparing threshold and lapse rate modulations in the entire population. Each dot is an individual animal, error bars are ±95% bootstrap CIs. Black points represent example rats. The majority of the population lies in the top left quadrant, showing co-modulations of both threshold and lapse rate parameters by history.

Figure 3:. History-dependent initial states capture comodulations in thresholds and lapse rates in the data

(A) Schematic of the model (accumulator with HISt) used to fit rat data in the Poisson Clicks task. (Top): Schematic of accumulation-to-bound model whose initial states are modulated by trial history producing history-dependent apparent lapses and threshold modulations. The model consists of sensory noise $\left({\sigma }_s^2\right)$ in click magnitudes, adaptation of successive click magnitudes based on an adaptation scale (ϕ) and timescale (τ_ϕ), accumulator noise $\left({\sigma }_a^2\right)$ added at each timestep, leak in the accumulator (λ), and decision bounds +/−B. We refer to this accumulator model with History-dependent Initial States as ‘HISt’ (Bottom): On κ fraction of trials, the model occasionally chooses a random action irrespective of the initial state and stimulus, with some bias (ρ) reflecting a motor errors or random exploration. These true lapses are not modulated by history, such that any history modulations arise from the initial states alone. (B) Model fits to individual rats: Psychometric data from 3 example rats conditioned on previous rightward (blue) or leftward (wins), overlaid on model-predicted psychometric curves (solid line) from the accumulation with HISt model. The three example rats were chosen to illustrate the diversity of history effects in the dataset, ranging from no history effects (left) - to history effects that largely created horizontal biases (center) and history effects that additionally affected lapse rates (right). (C): Psychometric curves (solid line) from the same example rats conditioned on model-inferred initial states (colors from pink to blue), showing a similar pattern to analytical predictions in Fig 1D. (D) Distribution of best fitting models for individual rats: Overall bar height for each model denotes the total number of rats for which that model scored the lowest BIC score. (E) Model comparison using BIC by pooling per trial BIC score across rats and computing mean. Lower scores indicate better fits. Mean of per trial BIC scores across rats were significantly lower for model with HISt (p = 9.85×10⁻¹⁸, paired t-test). Error bars are SEM. (F) Individual variations in history modulations captured by the accumulator model with HISt: History modulations of threshold parameters measured from psychometric fits to the raw data (x-axis) v.s. model predictions (y-axis). Individual points represent individual rats, point sizes indicate number of trials. The model captures a majority of the variability as evidenced by the points lying close to the unity line. (G) same as (F) but for history dependent lapse rate modulations. The model captures a majority of the variability in lapse rate modulations, implying that the magnitude of threshold and lapse rate modulations are coupled as predicted by our model, and that history-dependent initial accumulator states contribute to apparent lapses in this dataset.

Figure 4:. Model predictions about reaction times are borne out in data

(A) Schematic of reaction time task in rats, with similar structure to (Fig 2A), with two modifications: rats are allowed to break “fixation” anytime during the trial and make a choice, and are rewarded for choosing the side with the higher Poisson rate, encouraging longer sampling for more accurate estimates. (B) Average choice behavior on all trials (left) and following previous right or left wins (right) of 6 rats on the reaction time task (solid line), overlaid on individual rat behavior (translucent lines) (C) Average parameters (solid points) of history-conditioned psychometric curves, overlaid on individual parameters (translucent points) showing significant history modulations in threshold and lapse rate parameters (p < 0.05, Mann-Whitney U-test) (D-F) Reaction time signatures D) expected from accumulator models with no history dependence in initial states, E) expected from accumulator models with history dependent initial states and F) observed in data. First, error reaction times are expected to be shorter if initial states are history dependent, as seen in data (Left column, red curves are below green curves in E,F). Second, reaction times on trials following right wins are expected to be lower on rightward stimuli (positive half of x-axis), and similarly following left wins (Middle column, blue (pink) curves on the right (left) are lower than dotted lines in E,F). Finally, repetition biases in choices are expected to occur more frequently for short reaction times, when the effect of initial states is strong (Right column, curves are above dotted line for smaller RTs in E,F). (G) Joint fits of the accumulator model with history-dependent initial states to choices (left) and reaction times (right) of an example rat show good correspondence to data. Data represented by points (circles: choices, squares: reaction times) and model fits represented by lines (choices) or shaded bars (reaction times, thickness represents 95% bootstrap prediction intervals). Reaction times (right) are split by wins (green) or errors (red). (H) Scatter plot showing correspondence between history modulations in threshold (left) or lapse rate (right) parameters derived from data (x-axis) and model fits (y-axis). Individual points represent individual rats.