Cannabinoid Modulation of Dopamine Release During Motivation, Periodic Reinforcement, Exploratory Behavior, Habit Formation, and Attention

Abstract

Motivational and attentional processes energize action sequences to facilitate evolutionary competition and promote behavioral fitness. Decades of neuropharmacology, electrophysiology and electrochemistry research indicate that the mesocorticolimbic DA pathway modulates both motivation and attention. More recently, it was realized that mesocorticolimbic DA function is tightly regulated by the brain's endocannabinoid system and greatly influenced by exogenous cannabinoids-which have been harnessed by humanity for medicinal, ritualistic, and recreational uses for 12,000 years. Exogenous cannabinoids, like the primary psychoactive component of cannabis, delta-9-tetrahydrocannabinol, produce their effects by acting at binding sites for naturally occurring endocannabinoids. The brain's endocannabinoid system consists of two G-protein coupled receptors, endogenous lipid ligands for these receptor targets, and several synthetic and metabolic enzymes involved in their production and degradation. Emerging evidence indicates that the endocannabinoid 2-arachidonoylglycerol is necessary to observe concurrent increases in DA release and motivated behavior. And the historical pharmacology literature indicates a role for cannabinoid signaling in both motivational and attentional processes. While both types of behaviors have been scrutinized under manipulation by either DA or cannabinoid agents, there is considerably less insight into prospective interactions between these two important signaling systems. This review attempts to summate the relevance of cannabinoid modulation of DA release during operant tasks designed to investigate either motivational or attentional control of behavior. We first describe how cannabinoids influence DA release and goal-directed action under a variety of reinforcement contingencies. Then we consider the role that endocannabinoids might play in switching an animal's motivation from a goal-directed action to the search for an alternative outcome, in addition to the formation of long-term habits. Finally, dissociable features of attentional behavior using both the 5-choice serial reaction time task and the attentional set-shifting task are discussed along with their distinct influences by DA and cannabinoids. We end with discussing potential targets for further research regarding DA-cannabinoid interactions within key substrates involved in motivation and attention.

Keywords: adjunctive; attention; cannabinoids; dopamine; habit; motivation; reinforcement; timing.

FIGURE 1

Cannabinoids increase the frequency and amplitude of DA transients. Illustrative recording session in which the synthetic cannabinoid WIN was administered to an awake and freely moving rat. Stitched color plots [voltammetric current (z-axis) × applied scan potential (y-axis) × time (x-axis)] are shown above corresponding DA concentration traces. Vehicle (A), 0.2 mg/kg (B), and 0.8 mg/kg WIN (C) were administered in cumulative, ascending IV doses while FSCV measurements of DA release events occurred in the NAc shell in near real-time. Dose dependent increases in the frequency and amplitude of DA release events can be observed by the larger and more frequent green dots at a potential of +0.6 V in the color plots and the more frequent and pronounced transient peaks in the corresponding DA concentration traces. (D) WIN increased the frequency of DA release events but was less potent in chronically WIN-treated rats. A higher dose of WIN (0.8 vs. 0.2 mg/kg IV) was required to produce a significant increase in DA release vs. vehicle treated rats. (E) Heroin dose-dependently increased the frequency of DA release events but was less effective in chronically WIN-treated rats. In WIN-treated rats, heroin did not significantly increase the frequency of DA transients vs. vehicle at any dose tested. Republished from Gomez et al. (2020). (F) NAc–related functional connectivity in the left hemispheres. Shown are thresholded Z–score maps of functional connectivity for each group and each condition. Smoked THC reduced functional connectivity between the NAc and broad areas of the frontal, temporal, parietal and occipital lobes in occasional, but not chronic cannabis users. Republished from Gomez et al. (2020). ^∗p < 0.05.

FIGURE 2

Reinforcement schedules engender distinct behaviors and a depiction of two DA pathways. (A) The VI, FR, and FI schedules produce unique patterns of reinforced behavior. (B) The contingencies of reinforcement can produce adjunctive, goal-directed, or habitual behavior. (C) Illustrative projections associated with the nigrostriatal (dark red) and mesocorticolimbic (light red) DA pathways. DA in the NAc is thought to modulate converging input from brain regions including the amygdala (AMY), hippocampus (HIPP), and prefrontal cortex (PFC). Cortico-striatal loops are depicted in multicolor (orange-blue).

FIGURE 3

DA value signals encode price and modify the maximal price rats will pay for positive or negative reinforcement. Positive reinforcement: DA (DA) concentration (mean ± SEM) evoked by a reward predictive cue and delivery of a 45 mg sugar pellet decreased across the first five prices in a within-session behavioral economics-based task. In this task, the unit-price (responses/mg sugar) increased across fixed epochs of time (A). Optogenetic stimulation alters price sensitivity in a representative rat. Cumulative response records from one animal responding in the behavioral economic task under baseline conditions (light orange), and those in which DA release is amplified at cue presentation (dark orange) and at reward delivery (purple) (B). Changes in value were assessed using demand curves which measure changes in consumption in response to changes in unit-price. We formally extracted a dependent measure of value (i.e., α) which, represents the rate at which demand curve decay. Demand decays at a faster rate when the animal becomes more sensitive to price. As the animal is willing to pay less for the commodity, we would interpret the resulting increase in α as a decrease in value (C). The same data from the cumulative records in panel (B) are replotted in the form of demand curves to illustrate the optogenetic-induced shifts in value (D). Negative reinforcement: The concentration of DA evoked by a warning signal that predicted the opportunity to avoid decreased with the price to avoid. Inset: Representative avoidance trial shows that DA concentration began increasing in anticipation of warning signal presentation (E). The concentration of DA release events during the safety period decreased with price in trials in which the rat successfully avoided electrical foot shock (F). Optogenetic activation of VTA DA neurons at the warning signal made animals more sensitive to price, consistent with a negative reward prediction error (G). In contrast, optically stimulating DA neurons at successful avoidance made animals less sensitive to price, consistent with a positive reward prediction error (H). Republished from Schelp et al. (2017) and Pultorak et al. (2018). ^∗p < 0.05.

FIGURE 4

Cannabinoids modulate DA value signals during positive and negative reinforcement maintained under an FR schedule. Positive reinforcement: Systemically treating (intravenous; IV) rats with the cannabinoid receptor antagonist rimonabant increased the latency to respond for brain stimulation reward and decreased the concentration of cue-evoked DA value signals (A). Intrategmental infusions (IC) of rimonabant recapitulated these effects on reward seeking and DA release, demonstrating that eCB modulation of DA neural activity in the VTA is alone sufficient to modulate DA release and positive reinforcement (B). Systemically increasing 2AG levels by pre-treating rats with JZL184 (IV) reduced the latency to respond for brain stimulation reward and increased the concentration of cue-evoked DA value signals (C). Intrategmental infusions (IC) of JZL184 recapitulated these effects, suggesting that the action of 2AG in the VTA is alone sufficient to modulate DA release and positive reinforcement (D). Negative reinforcement: Systemic rimonabant administration (IV) reduced the number of successful avoidance responses and the concentration of DA evoked by the warning signal (E). Inhibiting DAGL-induced synthesis of 2AG by infusing THL into the VTA decreased avoidance and reduced 2AG tissue content in the VTA (F). Taken together, these observations generally support a DSI-model of 2AG-modulation of DA value signals (G) during positive and negative reinforcement maintained under an FR schedule. Republished from Oleson et al. (2012) and Wenzel et al. (2018). # < 0.05; **p < 0.001; ***p ≤ 0.001.

FIGURE 5

Cannabinoids modulate a temporally engendered pattern DA release during reinforcement maintained under an FI schedule and adjunctive behavior. An illustrative cumulative response pattern (top: raster plots; bottom: corresponding peri-event histograms) of a WIN-treated mouse responding for food reinforcement under a FI schedule. The pattern of lever pressing lawfully increases in the FI task to form a scalloped response pattern. The raster plot shows responses (black ticks) preceding food reinforcement (red triangle) across the 30 s interval. All trials are shown in chronological order as they occurred in a representative experimental session. The peri-event histogram shows the summation of responding under each corresponding raster plot. (A). WIN 55,212-2 accelerated the timing of scallop response pattern in a dose- and CB1-dependent manner. Mean behavioral response patterns following cannabinoid administration are plotted as a function of the interval duration (B). WIN amplified a temporally engendered pattern of DA release in a dose- and CB1-dependent manner. Mean DA concentration traces for each drug treatment conditions are plotted as a function of the interval (C). Cannabinoid-induced changes in interval timing were quantified by assessing the index of curvature—a computational measure of the extent and direction of change in the temporal response pattern produced by the FI schedule (D). WIN produced a negative index of curvature, suggesting an acceleration of timing behavior (E). Increasing 2AG with JZL184, but not increasing anandamide with URB597, accelerated interval timing (F,G). eCB-induced changes in reinforcement irrelevant or adjunctive behavior were assessed by quantifying responses on an inactive lever. Mean responses on the inactive lever initially increase before declining through the interval (H). JZL184 significantly decreased the percentage of time spent responding on the inactive lever, suggesting that adjunctive behavior was reduced by elevating 2AG levels (I). These data show that cannabinoids module periodically reinforced behavior and DA release under an FI schedule and, might suggest that a delicate balance of 2AG and DA release are necessary to produce the sweet-spot of intermittency that produces adjunctive behavior. Reproduced from Oleson et al. (2014).

FIGURE 6

CB1s are necessary for habit formation and adjunctive/exploratory behavior. To investigate the role of eCBs in habit formation and adjunctive behavior WT, CB1^+/–, and CB1^–/– were trained on a variable interval schedule and then tested in devaluation and exploration tests. (A) Normalized lever pressing during the valued versus the devalued condition for WT, CB1^+/–, and CB1^–/– mice. CB1 mutants showed sensitivity to sensory-specific satiety, suggesting that their actions were goal-directed rather than habitual. These data suggest that the CB1 and eCBs are necessary for habit formation. (B) Lever pressing (normalized) on the training lever versus a novel lever in WT, CB1^+/–, and CB1^–/– mice. Relative to other groups, CB1^–/– mice responded less on the novel lever, suggesting that the CB1 and eCBs may be involved in adjunctive behavior. (Republished from Hilário et al., 2007). (C) Graph shows responses in the valued (V) and devalued (DV) states in RI and RR training contexts. RR, random ratio (aka FR); RI, random interval (aka VI). During outcome devaluation procedures, control mice showed reduced lever pressing in the devalued state in the RR context but not the RI context. However, mice that lacked CB1s on OFC projection neurons into the striatum responded less in the devalued state under both RR and RI conditions. These data suggest that CB1s in cortical-striatal loops are necessary for habit formation (Republished from Gremel et al., 2016). Mice lacking the enzyme for the synthesis of 2AG from D1 MSNs (D1-Cre+) showed decreased exploration of a novel conspecific (D) and a novel environment (E). These data suggest that 2AG in the striatum plays an important role during adjunctive behavior (Republished from Shonesy et al., 2018). (F) Surprisingly, blocking metabolism of AEA with URB597 and 2AG with JZL184 disrupted rather than promote habit formation. These latter findings might suggest that AEA and 2AG are not important in habit formation or that non-specific behavioral effects (e.g., increased motivation for food) can confound tests of habitual behavior. **p < 0.01; ***p ≤ 0.001.

FIGURE 7

Cannabinoids modulate attentional processes. (A) Schematic of a single trial in the 5-choice serial reaction time task (5-CSRTT). Animals are trained to perform a nose poke when one of five cue lights is presented in any of the five nose-poke apertures following a fixed inter-trial interval (ITI), usually 4–6 s in length. Premature responses are tallied when made during this ITI. Responses made in one of the four nose pokes not illuminated is counted as an error of sustained attention. Reproduced from Cope et al. (2016). (B) Task schematics of two common set-shifting assessments for rodents. Panel (A) shows a schematic of the apparatus and examples of the stimuli used in the “dig” set-shifting task. Each pot has a unique odor (i.e., rose on left and citrus on right) and is filled with a unique digging medium (sequins on left, gravel on right). Only one stimulus feature is relevant to the location of a buried food reward in each phase of testing. Panel (B) shows a schematic of set-shift procedures performed in an operant version of the task. Rats are first trained to choose between two extended levers based on a light cue that is associated with one of the levers. After reaching criterion performance on that discrimination, there is an unsignaled change in rule and now the rat must ignore the light and choose levers based on their spatial location. Reproduced from Bizon et al. (2012). (C) Effect of CB1 antagonist/reverse agonist on impulsivity. Coadministration of WIN55,212-2 at 1.0 mg/kg (WIN1) prevents the effects of 3.0 mg/kg SR14716A (SR3) on inhibitory control in the 5-CSRTT. Reproduced from Pattij et al. (2007a). (D) Effect of acute THC administration on reversal learning. At 30 min before the start of the task, rats were administered vehicle, 0.01 mg/kg THC, or 1.0 mg/kg THC and the number of trials to reach criterion performance was recorded for a series of discriminations (SD, simple discrimination; CD, compound discrimination; Rev1,2,3, first, second, and third reversal stages; IDS, intradimensional shift; EDS, extradimensional shift). Animals in the 1 mg/kg THC treatment group exhibited marked deficits in performance at each of the reversal stages but not in the EDS stage. Reproduced from Egerton et al. (2005). (E) Altered compartmentalization of D2 immunogold stain in dendrites containing immunoperoxidase labeling for parvalbumin in the PL of the CB1^–/– mice. Cluster analysis reveals a significant change in compartmental distribution of D2 immunogold in parvalbumin dendrites of CB1^–/– mice. D2 immunogold density was assessed as particles of D2 immunogold/square μm dendritic area. In CB1^–/– mice relative to CB1^+/+ controls, a significant (p < 0.05) increase in D2 immunogold was observed in small dendrites, while a decrease in D2 immunogold per μm dendritic area was observed in medium parvalbumin dendrites in CB1^–/– mice relative to controls. Reproduced from Fitzgerald et al. (2012). *p < 0.01; **p ≤ 0.001.