Decision Making and Sequential Sampling from Memory

Abstract

Decisions take time, and as a rule more difficult decisions take more time. But this only raises the question of what consumes the time. For decisions informed by a sequence of samples of evidence, the answer is straightforward: more samples are available with more time. Indeed, the speed and accuracy of such decisions are explained by the accumulation of evidence to a threshold or bound. However, the same framework seems to apply to decisions that are not obviously informed by sequences of evidence samples. Here, we proffer the hypothesis that the sequential character of such tasks involves retrieval of evidence from memory. We explore this hypothesis by focusing on value-based decisions and argue that mnemonic processes can account for regularities in choice and decision time. We speculate on the neural mechanisms that link sampling of evidence from memory to circuits that represent the accumulated evidence bearing on a choice. We propose that memory processes may contribute to a wider class of decisions that conform to the regularities of choice-reaction time predicted by the sequential sampling framework.

Figure 1. Bounded evidence accumulation framework explains the relationship between choice and deliberation time

The decision is based on sequential samples of evidence until a stopping criterion is met, yielding a choice. A. Drift-diffusion with symmetric bounds applied to a binary decision. This is the simplest example of sequential sampling with optional stopping, equivalent to a biased random walk with symmetric absorbing bounds. The momentary evidence is regarded as a statistically stationary source of signal plus noise (Gaussian distribution; mean=μ, $\text{standard}\text{deviation}=\sqrt{\delta t}$) sampled in infinitesimal steps, δt. The resultant drift-diffusion process (noisy trace) is a decision variable that terminates at ±A (the bounds), to stop the process. If the termination is in the upper or the lower bound, the choice is for h₁ or h₂, respectively. B. Competing accumulators. The process is viewed as a race between two or more processes, each representing the accumulation of evidence for one of the choice alternatives. The architecture is more consistent with neural processes and has a natural extension to decisions between N>2 alternatives. The process in A is a special case, when evidence for h1 equals evidence against h₂ (e.g., Gaussian distributions with opposite means and perfectly anticorrelated noise). (reprinted from Gold & Shadlen 2007, with permission).

Figure 2. Similarities and differences among three types of decisions

Each is displayed as a series of events in time (left to right): stimuli are presented on a display monitor; the subject makes a binary decision, and, when ready, communicates the decision with a response. A. Perceptual decision. The subject decides whether the net direction of a dynamic, random-dot display is leftward or rightward. B. Value-based decision. The subject decides which of two snack items she prefers. The subjective values associated with the individual items are ascertained separately before the experiment. C. Decision from symbolic associations. The subject decides whether the left or right option is more likely to be rewarded, based on a sequence of shapes that appear near the center of the display. Each shape represents a different amount of evidence favoring one or the other option. In both A and C, the display furnishes more evidence with time (i.e., sequential samples), whereas in B, all the evidence in the display is presented at once. In A, sensory processes give rise to momentary evidence, which can be accumulated in a decision variable. Both B and C require an additional step because the stimuli alone don’t contain the relevant information. We hypothesize that the stimuli elicit an association or memory retrieval process to derive their symbolic meaning or subjective value as momentary evidence.

Figure 3. Memory contributions to value-based decisions are related to BOLD activity in the hippocampus, striatum, and vmPFC

Three tasks in humans use fMRI to assess brain regions involved in value-based decisions involving memory. A. Decisions based on transfer of reward value across related memories (following (Wimmer and Shohamy, 2012). In this “Sensory Preconditioning” task, participants first learn to associate pairs of stimuli with each other (e.g. squares with circles of different colors), without any rewards (Association phase). Next, they learn that one stimulus (e.g., the grey circle) leads to monetary reward, while another (e.g., the white circle) leads to no reward (Reward phase). Finally, participants are asked to make a decision between two neutral stimuli, neither of which has been rewarded before (e.g. blue vs. yellow squares; Choice phase). Participants often prefer the blue square to the yellow square or other neutral and equally familiar stimuli, suggesting they have integrated the reward value with the blue square because of the memory associating the blue square with the rewarded grey circle. The tendency to show this choice behavior is correlated with BOLD activity in the hippocampus and functional connectivity between the hippocampus and the striatum. These sorts of tasks allow experimenters to measure spontaneous memory-based decisions, without soliciting an explicit memory or rewarding it. In actual experiments, all stimuli are controlled for familiarity, likability, value, etc. B. Decisions about new food combinations involve retrieval of memories (following (Barron et al., 2013). In this task, foods are first evaluated separately (e.g. raspberries, avocado, tea, jelly, etc.). Then, participants learn to associate each food with random shapes (e.g. Asian characters; not shown here). Finally, participants are presented with a series of choices between two configurations of abstract shapes, which represent a new configuration of foods (e.g. raspberry-avocado shake vs. tea-jelly). These new choices, which involve retrieval and integration of two previously experienced stimuli, are correlated with activity in the hippocampus and in the vmPFC. C. Decisions about preferred snacks elicit retrieval of spatial memories (following (Gluth et al., 2015). After providing participants’ subjective preference for a series of snack items (not shown), participants learn a series of associations between snacks and a spatial location on the screen. Some associations are trained twice as often as others, creating memories that are relatively strong or weak. Participants are then probed to make choices between two locations, choices that require retrieval of the memory for the location-snack association. Choice accuracy and reaction times conform to bounded evidence accumulation and are impacted by the strength of the memory. Choice value on this task is correlated with BOLD activity in the hippocampus and in vmPFC. D. Overlay of regions in the hippocampus, striatum and vmPFC where memory and value signals were reported for the studies illustrated in A (red) B (green) and C (blue). Across studies, activation in the hippocampus, striatum and vmPFC is related to the use of memories to guide decisions. These common patterns raise questions about the neural mechanisms and pathways by which memories are used to influence value representations and decisions.

Figure 4. Putative neural mechanisms involved in updating a decision variable using memory

The symbols task (left) and, per our hypothesis, the snacks task (right) use memory to update a decision variable. The diagram is intended to explain why the updating of a decision variable is likely to be sequential in general, when evidence is derived from memory. In both tasks, visual information is processed to identify the shapes and snack items. This information must lead to an update of a decision variable represented by neurons in associative cortex with the capacity to represent cumulative evidence. This includes area LIP when the choice is communicated by an eye movement, but there are many areas of association cortex that are likely to represent the evolving decision variable. The update of the decision variable is effectively an instruction to increment or decrement the firing rate of neurons that represent the choice targets (provisional plans to select one or the other) by an amount, ΔFiringRate. The ΔFiringRate instruction is informed by a memory retrieval process, which is likely to involve the striatum. In the symbols task this is an association between the shape that is currently displayed and a learned weight (logLR value; Fig. 2C). In the snacks task it is likely to involve episodic memory, which leads to a value association represented in the vmPFC/OFC. Notice that there are many possible sources of evidence in the symbols task and potentially many more in the snacks task. Yet there is limited access to the sites of the decision variable (thalamo-cortical “pipe”). Thus, access to this pipe is likely to be sequential, even when the evidence is not supplied as a sequence. Anatomical labels and arrows should be viewed as hypothetical and not necessarily direct. PR, perirhinal cortex; HC, hippocampus.