Noradrenaline and dopamine neurons in the reward/effort trade-off: a direct electrophysiological comparison in behaving monkeys

Abstract

Motivation determines multiple aspects of behavior, including action selection and energization of behavior. Several components of the underlying neural systems have been examined closely, but the specific role of the different neuromodulatory systems in motivation remains unclear. Here, we compare directly the activity of dopaminergic neurons from the substantia nigra pars compacta and noradrenergic neurons from the locus coeruleus in monkeys performing a task manipulating the reward/effort trade-off. Consistent with previous reports, dopaminergic neurons encoded the expected reward, but we found that they also anticipated the upcoming effort cost in connection with its negative influence on action selection. Conversely, the firing of noradrenergic neurons increased with both pupil dilation and effort production in relation to the energization of behavior. Therefore, this work underlines the contribution of dopamine to effort-based decision making and uncovers a specific role of noradrenaline in energizing behavior to face challenges.

Keywords: choice; decision; locus coeruleus; motivation; pupil; substantia nigra.

Figure 1.

Experimental procedures. a, The task consists of squeezing a grip to obtain a reward. A trial started with a red point and the cue appeared within 900 ms after the animal initiated fixation. The trial was aborted immediately if the monkey broke fixation before reward delivery. After a variable delay of 1500 ± 500 ms, the dot turned green (Go signal), indicating that the monkey had to squeeze the bar strongly enough to reach the appropriate effort level, indicated by a blue point (feedback). If the monkey exerted enough force to score a correct response, it obtained the number of reward drops predicted by the cue. b, Experimental design and cue set used for the experiments: cues and background were isoluminant. Each trial corresponds to one of the nine conditions, defined by three levels of effort and three sizes of reward. c, Recording sites shown on MR images illustrating the position of electrodes targeting the SNc (orange dot) in Monkey E (left) and the LC (blue dot) in Monkey D (right). The middle image shows an example of simultaneous placement of recording electrodes in the LC (blue dot) and the SNc (orange dot). d, We used a pharmacological test to confirm our classification of SNc neurons as dopaminergic based on neurophysiological criteria. The firing of this SNc neuron fluctuates between 2 and 3 spikes/s before the intramuscular injection of apomorphine (0.05 mg/kg, i.m.). The injection induces a transient activation, presumably because of the arousing effect of the mechanical stimulation, and a lasting inhibition of about 20 min, presumably because of the stimulation of inhibitory autoreceptors on this neuron.

Figure 2.

Behavior in the task. a–c, Influence of trial number, effort level, and reward size on the choice to perform the trial for each of the three monkeys (Monkeys D, E, and A, respectively). After cue onset, monkeys could decide to abort the trial or to continue and exert the effort at the Go signal (choice to go). The influence of the three factors on the choice to go of each monkey was assessed using a logistic regression with a GLM. Parameter estimates are represented as the mean ± SEM of the regression coefficients across all sessions. For all three monkeys, the choice to perform the trial was influenced positively by the size of the expected reward (blue bar) and negatively by both the expected effort (red bar) and the progression through the session (black bar). The constant term was significant (green bar) for the three monkeys. d, Effect of expected effort on force production. Average force profile (±SEM) aligned on the action onset, broken down into the three effort levels: easy (yellow), intermediate (orange),and difficult (red). Monkeys used the information provided by the cues to adjust their behavior: they produced the minimum amount of force necessary to complete the trial (GLM, effort factor: β = 0.1, p < 10⁻¹⁰). e, Influence of effort on pupil area around the behavioral response. Average pupil diameter (±SEM) aligned on the response according to the three difficulty levels. f, Influence of experimental factors on the magnitude of pupil dilation response at the cue assessed using a GLM approach. Parameter estimates are represented as the mean ± SEM of the regression coefficients across all sessions. At cue onset, the pupil response was affected by the expected reward size, not by the expected effort or the progression through the session (trial number).

Figure 3.

Neuronal activity around events of interest. Neuronal activity around cue onset (a–d, vertical gray line) and action onset (e–h, vertical green line). a, Spike activity (raster and spike density function, orange line) of a SNc unit showing a strong activation at the cue. b, Sliding window analysis showing standardized firing rate of all SNc neurons recorded (z-scored by neuron, color-coded) around cue onset (t = 0). Each line represents the activity of a single neuron and neurons are sorted by increasing latency of the peak. c, Activity of a single LC neuron activated after cue onset (same representation as in a). d, Sliding window analysis of the activity of all LC neurons recorded, aligned around cue onset (same representation as in b). e, Activity of a representative SNc neuron around action onset. This cell shows a strong activation before the action and a more limited activation during the force production itself. f, Sliding window analysis of the population of SNc neurons around the action onset. The peak of the activation of SNc neurons can occur before or after the action onset. g, Activity of a single LC neuron around action onset. Note the clear double activation, with one peak before the action and one during the effort. h, Sliding window analysis of the population of LC neurons around action onset. Almost all LC neurons are activated both before and during the effort.

Figure 4.

Modulation of neuronal activity by reward and effort factors. a, Activity of SNc neurons at the cue. a1, Activity of a representative SNc unit (same representation as in Fig. 3a) across the nine conditions defined by three levels of effort (increasing from top to bottom) and the three levels of reward (increasing from left to right). The response magnitude increases with the reward and decreases with the expected effort. a2, Distribution of regression coefficients for reward (blue) and effort (red) based on a GLM analysis of cue evoked firing rates (from 0 to 400 ms after cue onset) for each SNc neuron of the population (n = 90). The distribution of the effort regression coefficients is significantly shifted toward negative values as indicated by the red point (mean) and horizontal bar (SEM) above the distribution. The distribution of the reward regression coefficients was shifted toward positive values, as indicated by the blue point (mean) and horizontal bar (SEM) above the distribution. a3, Averages of regression coefficients (solid bars) and SEM (error bars) across trials for all SNc neurons (n = 90) at the time of the cue onset. b, Activity of SNc neurons at the action. b1, Activity of the same representative SNc unit (same representation as in a1). It is clearly activated in relation to the action, but the response is equivalent across all conditions. b2, Distribution of the regression coefficients for action-related firing rates (from −100 to 400 ms from action onset) for each neuron of the SNc population (same representation as in b). The distributions of the reward and effort coefficients are both centered on zero, indicating that on average reward and effort do not modulate SNc activity during the action. b3, Averages of regression coefficients across trials for SNc neurons (n = 90) at the time of the action onset (same representation as in a3). c, Activity of LC neurons at the cue. c1, Activity of a representative LC unit across the nine conditions (same representation as in a1). The activation of this neuron increases with the reward, but there is no effect of effort. c2, Distribution of the regression coefficients of cue evoked firing rates for each LC neuron of the population (same representation as in a2). The distribution of the reward coefficients (in blue) is shifted toward positive values, whereas the distribution of effort coefficients is centered on zero. c3, Averages of regression coefficients across trials for LC neurons (n = 93) at the time of the cue onset (same representation as in a3). D, Activity of LC neurons at the action. d1, Same LC unit as in c1. The activation of this neuron increases with the amount of effort, both before and after action onset. d2, Distribution of the regression coefficients of action-evoked firing rates for each LC neuron of the population (same representation as in a2). The distribution of the effort coefficients (in red) is significantly shifted toward positive values, but the distribution of reward coefficients is centered on zero. d3, Averages of regression coefficients across trials for LC neurons (n = 93) at the time of the action onset (same representation as in a3). *p < 0.05; **p < 0.01.

Figure 5.

Population activity. a, Top, Population average activity (± SEM, shaded areas) of SNc neurons around the cue onset, broken down into the three reward conditions. The magnitude of the cue-evoked activation increases with the expected reward. a, Bottom, Population average (± SEM) of SNc neurons around cue onset, broken down into the three effort conditions. The magnitude of the activation decreases with the anticipated effort. b, Top, Population average SNc neurons around action onset sorted across the three reward conditions. The activity is not influenced by the expected reward size. b, Bottom, Population average of SNc neurons around action onset sorted across the three effort conditions. The firing increases with the expected effort, but only during the action itself. c, Top, Population average of all recorded LC neurons around cue onset sorted across the three reward conditions. The magnitude of the activation is greater in the high reward condition. c, Bottom, Population average of LC neurons around cue onset sorted across the three effort conditions. The activity shows little effect of effort. d, Top, Population average of LC neurons around action onset sorted across the three reward conditions. There is little difference across reward conditions. d, Bottom, Population average of LC neurons around action onset sorted across the three effort conditions. Action-evoked activity shows a strong positive modulation by the effort level both before and during the action.

Figure 6.

Neuronal activity and behavior. a, The influence of net value on choices is correlated with its influence on SNc cue activity (each dot represents an SNc neuron). Correlation between the regression coefficients of net value (reward size-effort level) on cue evoked activity of SNc neurons (trial-by-trial, ordinates) and the regression coefficients of net value on the choice to perform the trial (trial-by-trial, abscissa). There is a significant positive correlation between these two regression coefficients across SNc neurons (n = 90), indicating that the more SNc neurons are activated by the net value, the more likely the monkey is to engage in the trial. b, Trial-by-trial relationship between the firing of LC neurons at the action and the amount of exerted force (z-scored by session). Data were binned for display (800 trials per bin ± SEM, error bars). The activation of LC neurons strongly correlated with the amount of force produced during the action. c, Trial-by-trial relationship between the firing of LC neurons at the action and the magnitude of the pupil dilation response (z-scored by session). Data were binned for display (800 trials per bin ± SEM, error bars). The activation of LC neurons was strongly related to the action-related pupil dilation, which provides a proxy for the underlying autonomic activation. In all plots, lines represent robust regression fits.