Trial-history biases in evidence accumulation can give rise to apparent lapses in decision-making

Abstract

Trial history biases and lapses are two of the most common suboptimalities observed during perceptual decision-making. These suboptimalities are routinely assumed to arise from distinct processes. However, previous work has suggested that they covary in their prevalence and that their proposed neural substrates overlap. Here we demonstrate that during decision-making, history biases and apparent lapses can both arise from a common cognitive process that is optimal under mistaken beliefs that the world is changing i.e. nonstationary. This corresponds to an accumulation-to-bound model with history-dependent updates to the initial state of the accumulator. We test our model's predictions about the relative prevalence of history biases and lapses, and show that they are robustly borne out in two distinct decision-making datasets of male rats, including data from a novel reaction time task. Our model improves the ability to precisely predict decision-making dynamics within and across trials, by positing a process through which agents can generate quasi-stochastic choices.

Fig. 1. Trial history-dependent initial states give rise to apparent lapses.

a Schematic of two common suboptimalities: history biases (top) and lapses (bottom). (Left): Rat making one of two decisions (left, right) based on accumulated sensory evidence (clicks on either side). (Top left): History biases i.e. an inappropriate influence of the previous trial (n-1) on the current decision (n) in addition to sensory evidence. (Top right): Typically assumed effect of history bias on the psychometric curve, shifting it horizontally around the inflection point. (Bottom left): Lapses i.e. a tendency to make seemingly random choices irrespective of sensory evidence. (Bottom right): Typically assumed effect of lapses on the psychometric curve, vertically scaling its asymptotes (Figure adapted with permission from Bingni W. Brunton et al., Rats and Humans Can Optimally Accumulate Evidence for Decision Making. Science 340,95-98(2013). DOI:10.1126/science.1233912) b Normative model of within-trial processing. (Top) Optimal decision rule that chooses when the summed log-ratios of priors and likelihoods exceeds one of two decision bounds, corresponding to a drift-diffusion process. (Bottom left): Generative model, where one of two hypotheses (H1, H2) produce noisy evidence over time (ϵ_t). (Bottom right): A sample trajectory based on noisy evidence (bold line), and alternate trajectories (thin lines) based on noisy instantiations of the same drift rate (black arrow). c Model of across-trial processing that accommodates prior updates. Past choices and outcomes can affect the initial state with different magnitudes (η) and timescales (β) depending on whether they were wins/losses (top left/right). (Bottom): Example trial sequence ans corresponding initial states following previous wins (triangles) or losses (circles) on right (R) or left (L) choices. Colors denote initial state biases, towards positive (blue) or negative (pink) bounds. d Effect of initial state values on psychometric curves. Colors same as c. Small deviations in initial state (grey) lead to largely horizontal biases whereas larger deviations (saturated colors) additionally reduce its effective slope (dotted black lines) or “sensitivity" to stimulus. e Pooling psychometric function (black) across trials with different initial state biases gives rise to apparent lapses (purple arrow). Conditioning the curve on previous rightward (blue) or leftward (pink) wins reveals a modulation of apparent lapses by trial history.

Fig. 2. History-dependent threshold and lapse rate modulations in a large-scale rat dataset.

a Schematic of evidence accumulation task in rats: (Top): Phases of the ‘Poisson clicks’ task, including trial initiation in center port (left), evidence accumulation based on two streams of Poisson-distributed auditory clicks (middle) and choice report in one of two side ports followed by water reward for correct choices (right). (Bottom): Time-course of trial events in a typical trial. (Figure adapted with permission from Bingni W. Brunton et al., Rats and Humans Can Optimally Accumulate Evidence for Decision Making. Science 340,95-98(2013). DOI:10.1126/science.1233912) b Individual differences in history-dependence: Psychometric functions of three example rats from a large-scale dataset, displaying different kinds of history modulation. Choices are plotted conditioned on previous left (blue), right (pink) or all wins (black). (Left): Example rat with no history-dependence in choices, resembling the ideal observer. (Middle): Example rat with modulations of the threshold parameter alone, resembling the dominant conceptualization of history bias. (Right): Example rat with history-dependent modulation of both threshold and lapse rate parameter, similar to the majority of the population. Errorbars represent 95% binomial confidence intervals around the mean (n = [16946, 20577, 37523] trials for example 1, [8568, 9549, 18117] trials for example 2, [29358, 30821, 60179] trials for example 3 for psychometric curves conditioned on [right, left or all wins]) c Dataset displays significant modulations of both threshold and lapse rate parameters: Scatters showing parameters of psychometric functions following leftward wins (post left, blue) or rightward wins (post right, pink). Each pair of connected gray points represents an individual animal, solid colored dots represent average parameter values across animals. Trial history does not significantly affect the sensitivity parameter (top left) but significantly affects left, right lapse rate and threshold parameters (top right and bottom panels). (p = 0.8 for sensitivity, 3 × 10⁻¹⁷ for bias, 8 × 10⁻⁸ for left lapse, 6 × 10⁻⁷ for right lapse, two-sided Mann-Whitney U-test, n = 152) d Scatter comparing threshold and lapse rate modulations in the entire population (n = 152). Each dot is an individual animal, best-fit parameter values ± 95% bootstrap CIs. Black points represent example rats. The majority of the population lies in the top left quadrant, showing comodulations of both threshold and lapse rate parameters by history.

Fig. 3. History-dependent initial states capture comodulations in thresholds and lapse rates in the data.

a Schematic of the model used to fit rat data in the Poisson Clicks task. (Top): The model consists of trial history-dependent initial states (HISt) that can produce history-dependent apparent lapses and threshold modulations. Additionally, the model consists of sensory noise (${\sigma }_s^2$) in click magnitudes, adaptation of successive click magnitudes based on an adaptation scale (ϕ) and timescale (τ_ϕ), accumulator noise (${\sigma }_a^2$), leak in the accumulator (λ), and decision bounds +/–B. (Bottom): On κ fraction of trials, the model chooses a random action with some bias (ρ) reflecting motor errors or random exploration. These true lapses are not modulated by history, such that any history modulations arise from the initial states alone. (Figure adapted with permission from Bingni W. Brunton et al., Rats and Humans Can Optimally Accumulate Evidence for Decision Making. Science 340,95-98(2013). DOI:10.1126/science.1233912) b Model fits to individual rats: Psychometric data (mean accuracy ± 95% binomial confidence intervals) from 3 example rats conditioned on previous rightward (blue) or leftward (wins), overlaid on model-predicted psychometric curves (solid line) from the accumulation with HISt model. (n = [16946, 20577] trials for example 1, [8568, 9549] trials for example 2, [29358, 30821] trials for example 3 for psychometric curves conditioned on [right, left wins]) c: Psychometric curves (solid line) from the same example rats conditioned on model-inferred initial states (colors from pink to blue). d Distribution of best fitting models for individual rats: e Model comparison using BIC by pooling per trial BIC score across rats and computing mean (n = 152). Mean of per trial BIC scores across rats were significantly lower for model with HISt (p = 9.85 × 10⁻¹⁸, one-sided paired t-test) indicating better fits. Error bars are SEM. For individual data points see Supplementary Fig. 6f Individual variations in history modulations captured by the accumulator model with HISt: History modulations of threshold parameters measured from psychometric fits to the raw data (x-axis) v.s. model predictions (y-axis). Individual points represent individual rats (n = 152), point sizes indicate number of trials. g same as (f) but for history-dependent lapse rate modulations.

Fig. 4. Model predictions about reaction times are borne out in data.

a Schematic of reaction time task in rats (Figure adapted with permission from Bingni W. Brunton et al., Rats and Humans Can Optimally Accumulate Evidence for Decision Making. Science 340,95-98(2013). DOI:10.1126/science.1233912) b Average choice behavior on all trials (left; n = 223231 trials) and following previous right (n = 86109 trials) or left wins (n = 82678 trials; right) across 6 rats (solid line), overlaid on individual rat behavior (translucent lines). Errorbars represent 95% binomial confidence intervals around the mean. c Average parameters (solid points) of history-conditioned psychometric curves, overlaid on individual parameters (translucent points) showing significant history modulations in threshold and lapse rate parameters (p = 0.69 for sensitivity, 0.004 for threshold, 0.02 for left lapse rate, 0.02 for right lapse rate, two-sided Mann-Whitney U-test; n = 6). d–f Reaction time signatures (d) expected from accumulator models with no history dependence in initial states, (e) expected from accumulator models with history-dependent initial states and (f) observed in data (n = 223,231 trials across all stimulus strengths and rats). (Leftmost column) error reaction times are expected to be shorter if initial states are history-dependent. Red (green) represents RTs on errors (wins). (Middle column) reaction times on trials following right wins (blue) are expected to be lower on rightward stimuli (positive half of x-axis), and similarly following left wins (pink). (Rightmost columns) repetition biases in choices are expected to occur more frequently for short reaction times, when the effect of initial states is strong. Error bars represent SEM. g Joint fits of the accumulator model with history-dependent initial states to choices (left) and reaction times (right) of an example rat (n = 24413 trials). Data represented by points (circles: choices, mean accuracy ± 95% binomial confidence intervals; squares: reaction times, mean RT ± SEM) and model fits represented by lines (choices) or shaded bars (reaction times, thickness represents 95% bootstrap prediction intervals). Reaction times (right) are split by wins (green) or errors (red). h Scatter plot showing correspondence between history modulations in threshold (left) or lapse rate (right) parameters derived from data (x-axis) and model fits (y-axis). Individual points represent individual rats (n = 6), best-fit parameter values ± 95% bootstrap CIs.