Paradoxical evidence integration in rapid decision processes

Abstract

Decisions about noisy stimuli require evidence integration over time. Traditionally, evidence integration and decision making are described as a one-stage process: a decision is made when evidence for the presence of a stimulus crosses a threshold. Here, we show that one-stage models cannot explain psychophysical experiments on feature fusion, where two visual stimuli are presented in rapid succession. Paradoxically, the second stimulus biases decisions more strongly than the first one, contrary to predictions of one-stage models and intuition. We present a two-stage model where sensory information is integrated and buffered before it is fed into a drift diffusion process. The model is tested in a series of psychophysical experiments and explains both accuracy and reaction time distributions.

Figure 1. Reference Experiment.

Upper panel: A left or right offset vernier was presented for 10 ms followed by a variable blank background (ISI, here shown for 20 ms) and, then, by a second vernier for 10 ms. Lower panel: Observers were asked to indicate whether the first or second vernier was offset to the right. Performance improves quickly with increasing ISI, reaching ceiling performance at 50 ms.

Figure 2. One-stage and two-stage models of decision making.

(A) A vernier (stimulus ‘A’) is followed by a second vernier (stimulus ‘B’). The first vernier is either offet to the right (as shown) or to the left (not shown). The second vernier stimulus is always offset to the opposite side. Only one vernier is perceived and the offsets of the two vernier stimuli fuse. The perceived offset of the fused vernier is more strongly influenced by the second than the first vernier when the duration and of stimulus ‘A’ and ‘B’ are equal, . (B) One-stage model. After a sensory delay, the stimulus input is directly fed into the decision stage as the drift rate of a decision variable which is subject to a random walk. When the decision variable hits the upper boundary (), the decision is for the offset of the first vernier (stimulus ‘A’). When it hits the lower boundary (), the decision is for the offset of the second vernier (stimulus ‘B’). A motor response is executed accordingly. Variability in the drift leads to different reaction times (red and blue curves show reaction time distributions). It is important to note that observers push one button for left responses and one for right responses. In this figure, however, button ‘A’ is a symbol denoting responses according to the first vernier stimulus (either left or right) and button ‘B’ according to the second vernier stimulus. (C) Upper panel: After preprocessing and signal transmission of duration (sensory delay), the one-stage model translates the time course of the input directly into a time-varying drift rate of the decision process. Bottom panel: The time-varying drift rate directly drives the drift-diffusion process leading to trajectories which first increase and then decrease (orange trajectory,  = 10 ms; purple,  = 40 ms). The earlier the decision variable hits one of the boundaries, the faster the reaction times. For short (e.g. 10 ms) the trajectory does not reach any of the boundaries (, ) during stimulus presentation. One of the boundaries is reached after a random walk (orange line and reaction time distributions). For longer durations, the trajectory (purple line) more likely hits the upper than the lower boundary, leading to a decision for stimulus ‘A’. In few cases, a decision for stimulus ‘B’ is made because of the noise (purple reaction time distributions). (D) Experiment 1. In the psychophysical experiments, dominance is quantified as the percentage of responses which are in accordance with the first vernier. According to the one-stage model, vernier dominance increases when total stimulus duration increases (blue line), in stark contrast to the performance of human observers (green line; mean dominance across observers; error bars represent standard error of means, SEM). For the model, dominance is quantified as the percentage of trials in which the diffusion process hits the upper boundary (). (E) Two-stage model. The input is first integrated, before it is buffered and fed as a constant drift into the drift-diffusion process. (F) The input is delayed by and integrated with a leak. The value of the first stage is read out after stimulus termination , written into a buffer, and fed as a constant drift rate into the diffusion process at times greater than . Longer input durations lead to stronger negative drifts. Hence, the probability to hit the lower boundary increases with increasing vernier durations. (G) Performance of the two-stage model (purple line) is similar to the performance of human observers for total durations up to 80 ms (green circles, same human data as in D).

Figure 3. Experiment 2.

(A) A vernier (stimulus ‘A’) of duration is followed by a second vernier with opposite offset direction (stimulus ‘B’) of duration . (B) The longer the first vernier stimulus is presented, the stronger is its dominance (green circles). If first and second verniers are of the same duration, the second vernier dominates performance, i.e. performance is below 50% (dashed line). Relative vernier duration, , is plotted on the abscissa, mean vernier dominance across observers is plotted on the ordinate. The two-stage model (purple line) fits the psychophysical data well. (C) The 10% (downward pointing triangles) and 25% fastest responses (squares) vary only slightly with the relative vernier duration. The median (circles), the 75% quantile (diamonds) and 90% quantile show a strongly inverted U-shaped pattern (mean across observers). When either the first or the second vernier clearly dominate performance, response times are shorter than when first and second vernier are equally long (relative vernier duration 0.5). The two-stage model (purple lines) fits the psychophysical data well. (D) Mean response times across observers for the responses to the first vernier (red circles) and the second vernier (blue circles) show a similar pattern. The two-stage model captures this behavior well (solid lines). Error bars represent SEM.

Figure 4. Experiment 2, continued.

(A) Slow and fast responders in experiment 2. Box-plots of the reaction times for all 13 observers. A vernier stimulus was followed by a second vernier with opposite offset direction of 20 ms duration each. The lower and upper boundaries of the boxes represent the first and third quartile of the reaction time distribution. The median and its 95% confidence interval are indicated by the central line and the notch. Observers are ordered according to median response times. We separate observers in two groups. One group (green boxes) has median response times faster than 500 ms (dashed horizontal line), the other group slower than 500 ms (purple boxes). (B) Mean reaction time as a function of relative vernier duration for the first vernier (red symbols) and the second vernier (blue symbols) for fast responders (squares) and slow responders (circles). The solid lines represent the fit of the two-stage model. (C) Reaction time histograms of a typical observer showing responses to the first vernier (in red) and the second vernier (in blue). Responses are plotted for two stimulus conditions, where either the first vernier dominates (positive values; first vernier stimulus was presented for 32 ms followed by the second vernier of 8 ms) or the second vernier dominates (negative values; first vernier stimulus was presented for 8 ms followed by the second vernier of 32 ms). The solid lines are the corresponding two-stage model fits. The reaction time distributions for the other 12 observers are shown in the Supporting Figure S1. (D) The drift of the two-stage model (purple line) compared to the alternative two-stage model where the drift parameter was optimized for each stimulus condition independently (green circles). All other parameters are kept constant across different stimulus conditions but are different for each observer. Error bars represent SEM for both model variants.