Deliberative Decision-Making in Macaques Removes Reward-Driven Response Vigor

Abstract

Most of our daily decisions are governed by one of two systems: an impulsive system driving instantaneous decisions and a deliberative system driving thoughtful ones. The impulsive system reacts to immediately available concrete rewards. In contrast, the deliberative system reacts to more delayed rewards and/or punishments, which imposes consideration of longer-term choice consequences. Contingency management for addiction treatment is hypothesized to engage deliberative processes. Ultimately, in both decision-making situations, an action is needed to enact the decision. Whether those actions differ in implementation is an open question whose answer could inform as to whether distinct neural systems are engaged. To explore whether there is evidence of separate mechanisms between deliberated and immediate choices, we trained monkeys to perform a decision-making task where they made a choice on a touch screen between two visual cues predicting different amounts of reward. In immediate choice (IC) trials, the cues appeared at the final response locations where subjects could immediately touch the chosen cue. In deliberated choice (DC) trials, compound cues appeared orthogonally to the response locations. After a delay, allowing for decision formation, an identifying cue component was displaced to the randomly assigned response locations, permitting subjects to reach for the chosen cue. Both trial types showed an effect of cue value on cue selection time. However, only IC trials showed an effect of the competing cue on response vigor (measured by movement duration) and a reach trajectory that deviated in the direction of the competing cue, suggesting a decision reexamination process. Reward modulation of response vigor implicates dopaminergic mechanisms. In DC trials, reach trajectories revealed a commitment to the chosen choice target, and reach vigor was not modulated by the value of the competing cue. Our results suggest that choice-action dynamics are shaped by competing offers only during instantaneous, impulsive choice. After a deliberated decision, choice-action dynamics are unaffected by the alternative offer cue, demonstrating a commitment to the choice. The potential relevance to contingency management is discussed.

Keywords: addiction; choice; decision-making; deliberation; movement vigor.

FIGURE 1

(A) Subjects initiated a trial by pushing a blue led button. In DC trials (top timeline) two reward cues appeared, with the offer magnitudes 0–4 units of juice reward signaled by the blue bar height. Next, a circle or a diamond (response-symbol) was randomly assigned and added to each reward cue. Reward cues were removed during the delay period. Next, the response-symbols where repositioned randomly to the response locations allowing the subjects to reach for the symbol corresponding to their reward choice. The reward cue corresponding to the chosen symbol was shown for an additional 300 ms as visual feedback and reward was delivered. In IC trials (bottom timeline), subjects reached for the chosen reward cue as soon as it appeared. (B) Upper row shows the reward cues used with their corresponding amount of juice reward. Lower row shows 3 example trials with the corresponding reward distance (RD) between cues. (C) Choice performance across sessions (± SEM) as function of reward distance between the choice offer cues for different Chosen Values. Left/right panel from M1/M2.

FIGURE 2

(A) Cue selection time (time from when the cues were presented to when the monkey released the hold button) vs. reward distance (RD) between cues for different Chosen Values for DC (red curves) and IC (blue curves) trials for M1. (B) Same format as in (A) for M2. (C) Movement duration (time from when the monkey released the hold button to when the monkey touched the reward cue) vs. reward distance (RD) between cues for different Chosen Value for DC (red curves) and IC (blue curves) trials for M1. (D) Same format as in (C) for M2.

FIGURE 3

(A) Fixation probability on high/chosen (solid lines) and low/unchosen (dotted lines) cues vs. reward distance (RD) for different Chosen Values during DC trials, during the deliberation period. (B) Same format as in (A) for the delay period. (C) Same format as in B for cue selection and movement period. (D) Fixation probability on high/chosen (solid lines) and low/unchosen (dotted lines) cues for different Chosen Values vs. fixation order from end of the deliberation period. (E) Same format as in (C) for IC trials.

FIGURE 4

(A) Cue selection time vs. reward distance (RD) between cues for different Chosen Values for DC (red curves) and IC (blue curves) trials during the hand tracking sessions with subject M1. (B) Same format as in (A) for movement duration. (C) Same format as in B for maximum reach deviation. (D) Same format as in (C) for peak reach velocity.