A two-layered diffusion model traces the dynamics of information processing in the valuation-and-choice circuit of decision making

Abstract

A circuit of evaluation and selection of the alternatives is considered a reliable model in neurobiology. The prominent contributions of the literature to this topic are reported. In this study, valuation and choice of a decisional process during Two-Alternative Forced-Choice (TAFC) task are represented as a two-layered network of computational cells, where information accrual and processing progress in nonlinear diffusion dynamics. The evolution of the response-to-stimulus map is thus modeled by two linked diffusive modules (2LDM) representing the neuronal populations involved in the valuation-and-decision circuit of decision making. Diffusion models are naturally appropriate for describing accumulation of evidence over the time. This allows the computation of the response times (RTs) in valuation and choice, under the hypothesis of ex-Wald distribution. A nonlinear transfer function integrates the activities of the two layers. The input-output map based on the infomax principle makes the 2LDM consistent with the reinforcement learning approach. Results from simulated likelihood time series indicate that 2LDM may account for the activity-dependent modulatory component of effective connectivity between the neuronal populations. Rhythmic fluctuations of the estimate gain functions in the delta-beta bands also support the compatibility of 2LDM with the neurobiology of DM.

Figure 1

Drift diffusion model. The randomness of the path taken under the influence of noisy stimuli characterizes the diffusion models. A stimulus is represented in a diffusion equation by its influence on the drift rate of a random variable. This random variable, say the difference of evidence corresponding to the alternatives, accumulates the effects of the inputs over time until one of the boundaries is reached. The decision process ends when evidence reaches the threshold, and the time at which it occurs is called response time (RT). Response time (RT) depends on (a) the distance between the boundaries and the starting point, (b) the drift, that is, the rate at which the average (trend) of the random variable changes, and (c) the diffusion, that is, the variability of the path from the trend. These elements characterize the so-called drift diffusion model (DDM). The accumulation of evidence is then driven both by a deterministic component (drift) that is proportional to the stimulus intensity and by a stochastic component of noise (diffusion) that makes the evidence deviate from its own trend. The rationale of DDM is that, since the transmission and codification of the stimuli are inherently noisy, the quality of the feature extraction from such inputs may call for accumulation of a sufficient large sequence of the stimuli to get information [34]. Knowing the threshold level and the RT enables one to take a sight into the mechanism underlying the decision process [12, 88]. We can draw an analogy with a physical system and imagine the decisional process as the state of a “particle” moving within a potential well. Under this point of view, the persistence for relatively long periods of the state variable in the subthreshold area implies that the particle still entangled in the potential well enters an excited state where it remains for an exponentially distributed time interval with a certain decay time τd. If the combination of input and noise is sufficiently strong, then the particle is able to jump the barrier, that is, the threshold, and the system returns to an equilibrium state. The dynamics of the particle thus may resolve in a relaxation process [38] characterized by the oscillations between periods of subthreshold “disorder” inside the potential well and short impulses that trigger the system beyond the threshold in the rest state. This physical analogy allows better perception of how the DDM may fit the evolution of the input-output map underlying the neuronal model of the decision making process.

Figure 2

Example of binary encoding of information. The threshold value θ ₂ allows reading of the variable y as a binary code where the 1 s pulses occur when (y∣x) = g(N ₂) > g ₂(θ ₂). The lengths of the sequences of zeros provide the interpulse-intervals (IPI).

Figure 3

The two-layered diffusion model (2LDM) for decision making. Both stages (valuation and choice) are affected by noise. In the valuation stage the critical threshold indicates the firing rate of the neuronal populations involved, to which would correspond the expected reward. The outputs of this stage then are the differences between the responses of observed neuronal activity at the stimuli provided by the alternatives and the target. These measurements enter the next stage, where the decision is taken so as to optimize some utility criterion (reward). Hence, the attainment of the threshold in the decision stage indicates the preferred alternative. Feedback information flows from the decision stage in order to elicit the adaptation of the boundary in the valuation layer. In this way, a mechanism of reinforcement determines the competition between the alternatives and the valuation is biased to the most probable rewarded one.

Figure 4

Gain functions. In the plot are displayed the gain functions relative to the neuronal populations of the two layers. Both showed prominent rhythmic activity in the delta band. Increased oscillations up to beta band characterized population P ₂.

Figure 5

Time course of the correntropy coefficient between the phase signals in P ₁ and P ₂, (η), and between the surrogate phases (η _sur). Correntropy is a measure of nonlinear correlation that is obtained by the projection of the original vectors onto the reproducing kernel Hilbert space. The plot displays the time course of correntropy coefficients (η and η _sur) between phase signals φ ₁ and φ ₂ and between the corresponding surrogate phase signals (φ _sur1 and φ _sur2). Zero values correspond to independence between the signals.

Figure 6

Cumulative distribution function of distance between η and η _sur. The cumulative distribution function of the variable representing the difference between the correntropy coefficients η and η _sur is distributed as a Weibull-like random variable (with parameters a = 0.3752 and b = 1.5661).

Figure 7

Distance of the correntropy coefficients measured for the phase signals and their surrogates. Synchronized interaction between the two neuronal populations was determined in correspondence with the values of the correntropy distance vector above the critical value (0.756), which was calculated according to the distribution of a Weibull random variable with parameters a = 0.3752 and b = 1.5661 at the significance level of 5%. We observed a prominent asynchronous interaction.