Response outcomes gate the impact of expectations on perceptual decisions

Abstract

Perceptual decisions are based on sensory information but can also be influenced by expectations built from recent experiences. Can the impact of expectations be flexibly modulated based on the outcome of previous decisions? Here, rats perform an auditory task where the probability to repeat the previous stimulus category is varied in trial-blocks. All rats capitalize on these sequence correlations by exploiting a transition bias: a tendency to repeat or alternate their previous response using an internal estimate of the sequence repeating probability. Surprisingly, this bias is null after error trials. The internal estimate however is not reset and it becomes effective again after the next correct response. This behavior is captured by a generative model, whereby a reward-driven modulatory signal gates the impact of the latent model of the environment on the current decision. These results demonstrate that, based on previous outcomes, rats flexibly modulate how expectations influence their decisions.

Fig. 1. Auditory discrimination task and stimulus sequence statistics.

a Sketch of one trial of the task: cued by center port LED, rats poke in the center port to trigger the presentation of a mixture of two AM tones, each of which is associated with reward in the left (L) or right (R) port. Correct responses are rewarded with water, and incorrect responses are punished with a light plus a 5-s timeout. RT, reaction time. b, c Serial correlations in the sequence of stimuli were introduced by setting the probability of repeating the previous stimulus category P_rep (top in b) in blocks of 200 trials named repetitive block and alternating block (c). The stimulus strength s_k was randomly drawn in the kth trial (bottom in b) to yield the stimulus evidence e_k, that determined the distance to the categorization boundary, i.e., the discrimination difficulty of the stimulus (right in b). d The stimulus evidence e_k determined the distribution (top) from which the instantaneous evidence was drawn in each frame of the sound envelope (see color match with b). An instantaneous evidence trace (middle) and the AM modulated tones that result (bottom) are shown for an example stimulus with e = −0.48 (asterisks in b and d).

Fig. 2. Build-up and reset dynamics of the repeating bias.

a Psychometric curves for an example animal showing the proportion of rightward responses vs. stimulus evidence (left) or of repeated responses vs. repeating stimulus evidence (right) computed in the repetitive (blue dots) or alternating blocks (red dots; color code applies for all panels). This animal shows a block-independent rightward fixed side bias B > 0 (left), and a block-dependent repeating bias b matching the tendency of each block (right). Curves show fits using a probit function. b Proportion of repeated responses (median across n = 10 animals) computed in trials following a correct (left) or an incorrect response (right). c Repeating bias b versus fixed side bias B in the two blocks after a correct (left) or an incorrect response (right). Each pair of connected dots represents one animal. d Left: Fits of the proportion of repeated responses following trial sequences made of a different number of correct repetitions (blue gradient) or alternations (red gradient; see insets for color code). X⁺ and Y⁺ represent either rightward or leftward correct responses. E represents an error. Time in the sequences progresses from left to right. Right: same curves obtained when the sequence of correct repetitions is terminated by an error. e Repeating bias versus the length of the sequence of correct repetitions (left, blue) or alternations (right, red). Sequences terminated by an error are shown in black. Dark traces show median across animals while light traces show individual animals. Error bars show SD (a) or first and third quartiles (b, e).

Fig. 3. Dissecting two different history choice biases.

Cartoon of an example series of four choices, R⁺R⁺R⁺L⁺, illustrating the buildup of the lateral and transition biases. a The lateral bias, capturing the tendency to make rightward or leftward responses, increases toward the right in the first three R⁺ trials, and compensates this buildup with the last L⁺ response. Its net impact on the final trial is a rightward bias. b Schematic of the sequence of rewarded responses showing the transitions, defined as the relation between two consecutive responses, being repetitions (Rep, blue arrows) or alternations (Alt, red arrow). The animal computes each choice from combining its expectation based on a weighted sum of previous alternations (bottom gray balloon) and previous responses (upper gray balloon) with the current stimulus sensory information (see last trial). c Transition evidence z^T captures the tendency to repeat or alternate the previous response based on the series of previous transitions Rep⁺⁺Rep⁺⁺Alt⁺⁺ predicting a repetition in the final trial. d The transition bias γ^T is obtained by projecting the transition evidence z^T (c) onto the right–left choice space via a multiplication with the previous response r_t−1 (see gray arrow). e The evidence provided by the current stimulus is summed to the addition of the biases γ^L_t + γ^T_t and passed through a sigmoid function, yielding the probability of selecting a rightward response (Supplementary Fig. 5).

Fig. 4. Fitted weights quantifying the impact of the lateral and transition biases onto animals decisions.

Influence of past events on current choice when separately fitting the choices in trials after a correct (orange) or an error response (black). a GLM weights of previously rewarded (r⁺, left panel) and unrewarded (r⁻, right panel) responses. These kernels quantify the influence on choice of the side (left vs. right) of previous responses. b Weights of previous transitions (repetition vs. alternation) computed separately for T⁺⁺ (a rewarded trial followed by a rewarded trial), T⁻⁺ (error-rewarded), T⁺⁻ (rewarded-error), and T⁻⁻ (error-error). Strong positive weighting of T⁺⁺ transitions after-correct responses revealed that animals tended to reproduce previous repetitions and alternations between two consecutive rewarded trials. Points in a and b show median coefficients across animals (n = 10) and error bars indicate first and third quartiles. c Transition kernels for individual animals show the ubiquity across subjects of the reset of the kernel after errors. Some weights at trial lags 1 and 2 are not shown because of existing indeterminacies between regressors (see Supplementary Methods Section 2.3).

Fig. 5. Transition bias is reset after errors but accumulated transition evidence is maintained.

a, b Schematics showing three example traces of the transition accumulated evidence z^T_t (top) and transition bias on current response γ^T_t (bottom) in two hypothetical scenarios. To facilitate visualization, the pattern of responses in all example traces was R⁺R⁺R⁺ER⁺. a Complete reset hypothesis: after an error at t, both variables reset z^T_t+1 ≃ γ^T_t+1 ≃ 0. Evidence z^T_t+2 is then built up de novo, implying that biases before (γ^T_t) and after (γ^T_t+2) the reset are independent. b Gating hypothesis: after an error, evidence z^T_t+1 is maintained but it does not convert into a bias, leading to the reset γ^T_t+1 ≃0. After a correct response at t + 1, the conversion is recovered and the value γ^T_t+2 correlates with γ^T_t. In the example shown, this implies that the sorting of γ^T_t across traces is maintained for γ^T_t+2. c Transfer coefficient γ^T_t → γ^T_t+k versus trial lag k quantifies the degree to which the transition bias at trial t is predictive of the bias on subsequent trials (blue dashed boxes in b). It is calculated separately depending on the outcome of each trial (colored lines show rewarded choices and black lines error choices; see the section 2.6. in Supplementary Methods for details). While the transfer coefficient vanishes after errors (i.e., reset of the bias; black dots), a correct response following an error (light orange) brings it close to the value obtained when there are no errors (dark orange dots). This implies that the information about the value of the bias γ^T_t is maintained when the bias is reset (i.e., gating hypothesis).

Fig. 6. Dynamic generative model of history-dependent perceptual decisions.

a Architecture of the model. The sensory module accumulates the instantaneous stimulus evidence of the current trial. The lateral module maintains the lateral evidence z^L, which is updated depending on the last response and its outcome (updates Δ_L). The transition module maintains the transition evidence z^T, which is updated depending on the last transition (i.e., ++,−+,+−,−−; updates Δ_T). The modulatory signal c^T is updated based on the last outcome (updates Δ_C). The transition bias is obtained from the product  γ^T = z^T × c^T × r_t−1. The sum of sensory evidence and the lateral and transition biases determined the probability to choose either response at the current trial. Parameters were fitted to the choices of each rat separately. b–e Best-fitting values of the model parameters. Bars show median across seven rats (black and blue points). Three rats were excluded from the statistics because the fitted model yielded a solution without gating dynamics (gray points). b Lateral evidence outcome-dependent update Δ_L. c Outcome-dependent leak of the lateral bias λ_L. d Transition evidence update Δ_T, depending on the outcome of the last two trials (++, −+, +−, −−). e Outcome-dependent leak of the transition bias λ_T. f Outcome-dependent update of the transition gating signal Δ_C. A value of −1 corresponds to an extinction of the gating signal on the subsequent trial (i.e., a full blockade of the corresponding bias), while +1 corresponds to full recovery of the bias (i.e., gating equal to its maximum value of 1). g Example traces of the dynamics of the latent variables across 25 trials switching from a repetition to an alternation block (fitted parameters correspond to rat 12 shown as blue points in b–e). Traces depict the variables stimulus evidence S, z^L, z^T, c^T, and overall probability to choose a rightward response. Symbols on the corresponding trial axis represent inputs to the variables z^L, z^T, and c^T: left (green) vs. right (purple) responses; repeating (blue) vs. alternating (red) transitions; and rewarded (orange) vs. error (black) outcomes. Symbols shape represent different outcome combinations (see inset). Notice the reset of c^T after errors and the maintenance of z^T afterwards (asterisks).

Fig. 7. Generative model simulation compared to experimental data.

Comparison between experimental data (dots) and model simulation (black curves) showing the repeating bias b for different trial sequences. In the model, b was decomposed into the transition bias contribution (blue curves) and lateral bias contribution (green curves). a Repeating bias b versus number n of correct repetitions (top) or alternations (bottom). Data are the same as the color curves in Fig. 2e. b Repeating bias versus n after a repetitive (top) or alternating sequence (bottom) terminated by an error (as black curves in Fig. 2e). Notice the different range in the b axes compared with a. c Repeating bias for sequences with an error E flanked by correct repetitions (top) or alternations (bottom). The bias b is given as a function the trial distance to the error response (distance zero represents b after the error). d Repeating bias for all sequences made of n ≤ 8 repetitions (R) and alternations (A). Top panel shows correct sequences while bottom panel shows correct sequences terminated by an error. In all panels, data and model show median across n = 10 rats. Error bars and shaded areas show first and third quartiles.