Value-based decision making via sequential sampling with hierarchical competition and attentional modulation

Abstract

In principle, formal dynamical models of decision making hold the potential to represent fundamental computations underpinning value-based (i.e., preferential) decisions in addition to perceptual decisions. Sequential-sampling models such as the race model and the drift-diffusion model that are grounded in simplicity, analytical tractability, and optimality remain popular, but some of their more recent counterparts have instead been designed with an aim for more feasibility as architectures to be implemented by actual neural systems. Connectionist models are proposed herein at an intermediate level of analysis that bridges mental phenomena and underlying neurophysiological mechanisms. Several such models drawing elements from the established race, drift-diffusion, feedforward-inhibition, divisive-normalization, and competing-accumulator models were tested with respect to fitting empirical data from human participants making choices between foods on the basis of hedonic value rather than a traditional perceptual attribute. Even when considering performance at emulating behavior alone, more neurally plausible models were set apart from more normative race or drift-diffusion models both quantitatively and qualitatively despite remaining parsimonious. To best capture the paradigm, a novel six-parameter computational model was formulated with features including hierarchical levels of competition via mutual inhibition as well as a static approximation of attentional modulation, which promotes "winner-take-all" processing. Moreover, a meta-analysis encompassing several related experiments validated the robustness of model-predicted trends in humans' value-based choices and concomitant reaction times. These findings have yet further implications for analysis of neurophysiological data in accordance with computational modeling, which is also discussed in this new light.

Fig 1. Task.

(a) For all studies, subjects were required to make a two-alternative forced choice between a pair of randomly sampled foods with uncorrelated subjective values. The original data set to which the forthcoming computational models were fitted was distinguished by a paradigm with adjacent stimuli and persistent fixation, allowing for only covert shifting of the focus of visual attention. (b) In contrast, the other studies included in the meta-analysis featured stimuli that were well separated spatially and thus required eye movements.

Fig 2. Dynamical models of neural decision making.

(a) The race model [–15] is the most basic of these by virtue of assuming that the representations of each option are completely independent. (b) Input-dependent competition is the signature feature common to the subtractive normalization-or-feedforward-inhibition (SNFI) model [34,35], the divisive normalization-or-feedforward-inhibition (DNFI) model [–40], and the neural drift-diffusion (NDD) model [,–11]. The NDD model is nested within the SNFI model but instead posits perfect competition (i.e., i_v = 1). (c) The competing-accumulator (CA) model [26,36] is instead characterized by state-dependent competition via lateral inhibition at the level of accumulating decision signals. (d) The subtractive competing-accumulator (SCA) and divisive competing-accumulator (DCA) models take a novel approach of including both input-dependent competition and state-dependent competition in tandem. Solid green and dashed red arrows indicate excitatory and inhibitory connections, respectively. At the level of value signals, the leftmost vertical and diagonal dashed red arrows denote lateral inhibition (i.e., input normalization or relative coding) and feedforward inhibition, respectively, which are represented collectively here because in this context they are equivalent in terms of output. The gray clouds reflect independent sources of noise. Vertical gray bars symbolize thresholding mechanisms. v_x represents the ensemble of value-encoding neurons representing alternative x. d_x represents the corresponding ensemble of decision-making neurons. e_x represents the corresponding ensemble of execution neurons. The free parameters are b for baseline input, g for the gain of value-signal inputs, σ for noise, i_v for value-signal inhibition as part of a subtractive transformation, s for semisaturation as part of a divisive transformation, and i_d for decision-signal inhibition.

Fig 3. The supralinear subtractive competing-accumulator (SSCA) model.

The SSCA model builds upon the SCA model with the intention of approximating the net effects of the addition of an attentional module that selectively modulates value signals. The positive-feedback loops that are consequently formed generate disproportionate amplification of value signals that are already greater in magnitude, thus promoting “winner-take-all” processing [32]. This schematic only depicts a positive-feedback loop at the level of value signals to adhere more closely to the parsimonious implementation used here with a static supralinear power law requiring only one free parameter, a. However, also plausible are loops at the next level bridging decision-making signals and attentional processes either with or without intermediate value signals. The contrast between solid and dotted green lines symbolizes the asymmetry in the positive-feedback loop’s impact on each alternative’s representation. As time progresses, there is an increasingly higher probability of attention being directed at the alternative with greater value, which is denoted by the G subscript, rather than the alternative with lesser value, which is denoted by the L subscript.

Fig 4. Model comparison.

(a) The global fitting performance of each candidate model is first shown for the training data set. The χ² statistic corresponds to raw lack of fit, but two levels of adjustment for model complexity are also provided in the form of the corrected Akaike information criterion (AICc) and the Bayesian information criterion (BIC). (b) A test data set of equal size was reserved for out-of-sample validation. The saturated model revealed the best out-of-sample performance possible with maximal degrees of freedom. Degrees of freedom are listed in parentheses.

Fig 5. Choice accuracy.

(a) Choice accuracy (i.e., the probability of correctly choosing the option with greater value) as a function of both values is displayed first for the empirical data set. Only the probabilities of correct choices are provided in the upper-left corners of each panel to avoid redundancy. (b) Accuracy is likewise shown for data sets simulated with each of the computational models in the first and third rows. Differences between model predictions and observed results are highlighted in the second and fourth rows. (c) The differences between chosen and nonchosen values and their sums are provided for reference.

Fig 6. Reaction time.

(a) Following the conventions of the previous figure, mean reaction time (RT) as a function of both values is displayed first for the empirical data set. (b) RT is likewise shown for data sets simulated with each of the computational models in the first and third rows. Differences between model predictions and observed results are highlighted in the second and fourth rows. (c) The differences between chosen and nonchosen values and their sums are again provided for reference. The upper-left and lower-right corners of each panel correspond to correct and incorrect choices, respectively, and the diagonal midline between them corresponds to indifferent choices.

Fig 7. Reaction-time distributions.

RT distributions for each combination of chosen (“C”) and nonchosen (“N”) values are displayed with 100-ms bins for the empirical data set (bars) and the data set generated by the preferred SSCA model (lines).