Adaptive History Biases Result from Confidence-Weighted Accumulation of past Choices

Abstract

Perceptual decision-making is biased by previous events, including the history of preceding choices: observers tend to repeat (or alternate) their judgments of the sensory environment more often than expected by chance. Computational models postulate that these so-called choice history biases result from the accumulation of internal decision signals across trials. Here, we provide psychophysical evidence for such a mechanism and its adaptive utility. Male and female human observers performed different variants of a challenging visual motion discrimination task near psychophysical threshold. In a first experiment, we decoupled categorical perceptual choices and motor responses on a trial-by-trial basis. Choice history bias was explained by previous perceptual choices, not motor responses, highlighting the importance of internal decision signals in action-independent formats. In a second experiment, observers performed the task in stimulus environments containing different levels of autocorrelation and providing no external feedback about choice correctness. Despite performing under overall high levels of uncertainty, observers adjusted both the strength and the sign of their choice history biases to these environments. When stimulus sequences were dominated by either repetitions or alternations, the individual degree of this adjustment of history bias was about as good a predictor of individual performance as individual perceptual sensitivity. The history bias adjustment scaled with two proxies for observers' confidence about their previous choices (accuracy and reaction time). Together, our results are consistent with the idea that action-independent, confidence-modulated decision variables are accumulated across choices in a flexible manner that depends on decision-makers' model of their environment.SIGNIFICANCE STATEMENT Decisions based on sensory input are often influenced by the history of one's preceding choices, manifesting as a bias to systematically repeat (or alternate) choices. We here provide support for the idea that such choice history biases arise from the context-dependent accumulation of a quantity referred to as the decision variable: the variable's sign dictates the choice and its magnitude the confidence about choice correctness. We show that choices are accumulated in an action-independent format and a context-dependent manner, weighted by the confidence about their correctness. This confidence-weighted accumulation of choices enables decision-makers to flexibly adjust their behavior to different sensory environments. The bias adjustment can be as important for optimizing performance as one's sensitivity to the momentary sensory input.

Keywords: adaptation; choice history bias; confidence; perceptual decision-making.

Figure 1.

Quantifying choice history bias and behavioral task. A, B, Behavioral tasks. Observers judged the net direction (up vs down) of a dynamic random dot pattern of variable direction and coherence. A, Experiment 1, decoupling choice and motor response. After a blank fixation interval, a choice-response Mapping cue was shown before (Pre) or after the presentation (Post) of the motion stimulus, which also varied in duration. Observers responded after dot motion offset in the Pre-condition and after Mapping cue offset in the Post-condition. Auditory feedback was provided after incorrect responses. B, Experiment 2, manipulating stimulus repetition probabilities. Left, Random-dot motion and fixation cross were shown throughout the trial. A beep indicated the onset of the evidence interval, which contained some level of coherent motion (0% on some trials). A second beep indicated the evidence offset and start of the response interval (deadline: 3 s). Right, Three repetition probabilities between motion directions across trials yielded three environmental conditions: Neutral (repetition probability of 0.5), Repetitive (repetition probability of 0.8), and Alternating (repetition probability of 0.2). C, Signed motion coherence levels (cyan) and categorical choices (purple) from a sequence of 15 trials recorded in Neutral in Experiment 2. Positive values of stimulus intensity correspond to upward motion and negative ones to downward motion. D, Psychometric functions conditioned on previous choice in Neutral exhibit history biases in three example participants. See main text for details.

Figure 2.

Stronger impact of previous choice than of previous motor response on current bias. A, Impact of previous choices and motor responses as a function of lag. B, As in A, but for impact of previous stimulus categories. Shaded areas, SEM. •p < 0.05 (FDR-corrected t test) across participants.

Figure 3.

Adjustment of choice history biases to environmental statistics. A, Stimulus repetition probabilities for Repetitive, Alternating, and Neutral. In Repetitive, repetition of the motion direction from two trials back could occur due to a sequence of two repetitions or two alternations (probability: 0.8 × 0.8 + 0.2 × 0.2 = 0.68). In Alternating, the probability of repetition of the same direction oscillated ∼0.5 as a function of lags, with decreasing deviation from 0.5. B, Impact of previous stimuli and choices on current choice for Lag 1. Green dots and blue triangles indicate single observers. Grey lines connect values from both conditions for each single observer. Arrows indicate changes of group mean weights from Neutral (red cross) during Repetitive and Alternating, respectively. C, Psychometric functions conditioned on previous choice (group average). Left: Repetitive, leftward shift from dashed to dotted line corresponding to a bias to repeat the previous choice. Right: Alternating, leftward shift from dotted to dashed line indicating a bias to alternate the previous choice. D, Correct weights as functions of lags in Repetitive and Alternating. E, Adaptivity indices (correlation coefficient with the history template) computed from correct kernels from the Repetitive and Alternating conditions. Dotted line, Correlation between history templates for Repetitive and Alternating; shaded areas, SEM. •p < 0.05 (FDR-corrected t test) across participants. *p < 0.05, ****p < 0.0001.

Figure 4.

No differences in history weights for synthetic observers without bias adjustment. Results of two simulations of synthetic observers without bias adjustment analyzed as real observer data for Figure 3, B and D. A, Synthetic observers with all parameters taken from the real observers' estimates for Repetitive and Alternating, but history weights set to 0. B, Synthetic observers with all parameters taken from the real observers' estimates for Neutral. Note the difference in the y-axis scale between these simulated observers and the real observer data in Figure 3. Shaded areas, SEM across synthetic observers.

Figure 5.

Behavioral performance depends on bias adjustment and perceptual sensitivity. A, Correlation between adaptivity index and proportion of correct choices. Left, Repetitive; middle, Alternating; right, Neutral. Insets, Correlations for the simulated observers without bias adjustment (compare Fig. 4A). B, Correlation between sensitivity (i.e., slope of the psychometric function) and the proportion of correct choices. Left, Repetitive; middle, Alternating; Right, Neutral.

Figure 6.

Modulation of bias adjustment by proxies of confidence in previous choice. A, Scaling of model-based confidence and uncertainty with evidence strength on correct and error trials. Adapted from Urai et al. (2017) under a CC-BY 4.0 license. B, Difference between previous choice weights from Repetitive and Alternating, sorted by previous choice correctness and coherence. C, Comparison between previous correct and incorrect weights, for Repetitive and Alternating. Weights were first calculated separately for each previous coherence level and then pooled across coherence. D, Reaction time as function of motion coherence sorted by correctness (pooled across Repetitive and Alternating). E, Difference between previous RTconf × correct modulation weights from Repetitive and Alternating, sorted by previous coherence. See main text for details of the multiplicative modulation model. F, RTconf × correct weights for Repetitive and Alternating. Modulation weights were first calculated separately for each previous coherence level and then pooled across coherence. Shaded areas, SEM. •p < 0.05 (FDR-corrected t test) across participants. *p < 0.05, **p < 0.01, ***p < 0.001, ****p < 0.0001.