Cell-type and projection-specific dopaminergic encoding of aversive stimuli in addiction

Abstract

Drug addiction is a major public health concern across the world for which there are limited treatment options. In order to develop new therapies to correct the behavioral deficits that result from repeated drug use, we need to understand the neural circuit dysfunction that underlies the pathophysiology of the disorder. Because the initial reinforcing effects of drugs are dependent on increases in dopamine in reward-related brain regions such as the mesolimbic dopamine pathway, a large focus of addiction research has centered on the dysregulation of this system and its control of positive reinforcement and motivation. However, in addition to the processing of positive, rewarding stimuli, there are clear deficits in the encoding and valuation of information about potential negative outcomes and how they control decision making and motivation. Further, aversive stimuli can motivate or suppress behavior depending on the context in which they are encountered. We propose a model where rewarding and aversive information guides the execution of specific motivated actions through mesocortical and mesolimbic dopamine acting on D1- and D2- receptor containing neuronal populations. Volitional drug exposure alters the processing of rewarding and aversive stimuli through remodeling of these dopaminergic circuits, causing maladaptive drug seeking, self-administration in the face of negative consequences, and drug craving. Together, this review discusses the dysfunction of the circuits controlling different types of aversive learning as well as how these guide specific discrete behaviors, and provides a conceptual framework for how they should be considered in preclinical addiction models.

Keywords: Associative learning; Cocaine; Nucleus accumbens; Prefrontal cortex; Punishment; Self-administration.

Figure 1:. Context-specific control of behavior by rewarding and aversive stimuli.

While stimuli such as rewards or footshocks are often defined as rewarding or aversive, it is important to note that their effects on behavior are dependent on the context in which they are presented. Here we outline four different operant behavioral paradigms: positive reinforcement, positive punishment, negative reinforcement, and negative punishment in which the same stimulus can increase or decrease behavior. (A) Positive reinforcement is defined by the presentation of a stimulus increasing the probability that subsequent behavior will occur, i.e. a lever press results in food delivery. (B) Positive punishment results when the presentation of a stimulus results in a reduction in behavioral responding, i.e. a lever press results in a contingent aversive effect, such as footshock. (C) Negative reinforcement is defined by the removal of an aversive stimulus, which increases the probability of subsequent behavior, i.e. a lever press turns off a foot shock. (D) Negative punishment reduces behavioral responding because the behavioral response (lever press) results in removal of a positive stimulus, such as food. The difference between these behaviors is critical in understanding how neural circuit responses to environmental stimuli encode information.

Figure 2.. Behavioral paradigms for understanding passive and active behavioral responses to aversive stimuli

Here we outline four different aversive learning paradigms: fear conditioning, conditioned place aversion (CPA), negative reinforcement, and punished responding. (A) Fear conditioning is a Pavlovian model for passive aversive learning. A foot shock is used as an unconditioned stimulus (US) to elicit a fear response such as freezing, and a tone or other neutral cue is the conditioned stimulus (CS). After several sessions of pairing a tone with foot shock, mice learn to associate the tone with fear and freeze in response to presentation of the tone itself. (B) CPA is another passive behavioral model of fear learning used to study drug dependence and withdrawal. It uses an apparatus with two to four chambers, each characterized by different contextual cues. During the conditioning phase, individual chambers are paired with an aversive stimulus or a neutral stimulus. Then, on test day, animals are allowed access to all chambers and demonstrate preference/avoidance of one of the chambers, depending on which stimuli it was paired with. (C) Negative reinforcement is an operant behavioral paradigm in which animals lever press or nose poke to remove or prevent the presentation of an aversive stimulus, such as a foot shock. (D) Punished responding is a task where animals are trained to respond for the presentation of a reward. During punishment trials, responses result in both presentation of reward and delivery of an aversive stimulus, such as a foot shock. Punishers reduce response rates as animals learn to omit responding to avoid the aversive stimulus.

Figure 3:. Neural circuits implicated in different aspects of context-dependent aversive learning.

Complex networks of brain regions in the mesocorticolimbic dopaminergic system are involved in the encoding of information and behavioral responses in various aversive learning paradigms. (top) CPA: Noxious stimuli recruit the LHb, which signals to the mesolimbic dopaminergic system through the VTA and RMTg. Increased dopamine release in the mPFC and decreased dopamine in the NAc mediate avoidance behavior. (middle) Fear conditioning: Although similar brain regions as those involved in CPA are activated, unavoidable fearful stimuli drive freezing behavior through the additional recruitment of the amygdala, especially the BLA and CeA. (bottom) Negative reinforcement: Active avoidance behavior and relief responding occurs through increased activation of the NAc and inhibition of the CeA. BLA, basolateral amygdala; CeA, central amygdala.

Figure 4:. Proposed model for the dysregulation of dopaminergic circuits mediating aversive learning in addiction.

(top right) Naive: In naive animals, glutamatergic projections (green) from the LHb and GABAergic projections (red) from the RMTg modulate the activity of the VTA (Jhou et al., 2009; Lammel et al., 2012, 2014). The VTA projects to the mPFC and NAc (Morales & Margolis, 2017), and through dopaminergic signaling gates encoding of information about salient stimuli and associated cues (Phillips et al., 2003; Wanat et al., 2009). Additionally, glutamatergic neurons projecting from the BLA, hippocampus (Hipp), and PFC synapse on both D1 and D2 containing MSN populations in the NAc; Hipp inputs show stronger synaptic strength at D1 MSNs, relative to D2 MSNs, at baseline (Sjulson et al., 2018; MacAskill et al., 2014; Pascoli et al., 2014) (bottom right). Chronic drug exposure: Following chronic drug exposure, RMTg inhibitory regulation of the VTA is increased (Maroteaux and Mameli, 2012). Currently, it is unclear how projections from the LHb to VTA, and LHb to RMTg are altered by chronic drug exposure - denoted by a question mark. VTA to NAc projection (dotted) results in reduced dopamine release in the NAc both at baseline and in response to cocaine-associated stimuli (Calipari et al., 2012, Ferris et al., 2012; Willuhn et al., 2014). Basal dopamine levels in the mPFC are reduced following chronic drug exposure (Parsons et al., 1991). However, it is important to note that they are potentiated in response to drug-associated stimuli (Parsegian & See, 2014). This dysregulation of the dopaminergic system likely contributes to dysfunctional aversive learning across a number of modalities. (left inset) D1 and D2 medium spiny neurons in the NAc: Chronic drug exposure dysregulates the encoding of these dopaminergic signals downstream via cell-type specific remodeling of MSN inputs in the NAc. D1-mediated signals are enhanced both at baseline and in response to cocaine-associated cues (Calipari et al., 2016). The enhancement of BLA and Hipp inputs to D1 MSNs, could underlie these effects (Lee et al., 2013; MacAskill et al., 2014; Pascoli et al., 2014). There is conflicting evidence in regard to mPFC inputs directly onto D1s, with a majority of reports finding reductions or no change in AMPAR/NMDAR ratios (MacAskill et al., 2014, Joffee & Grueter 2016); however, Pascoli et al., 2014 observed reductions in AMPAR/NMDAR ratios as well as increases in rectification index and EPSCs, suggesting that the input could be increased in regard to some forms of plasticity. Innervation onto D2-type MSNs by the Hipp have been reported to be enhanced (Sjulson et al., 2018), however changes in in vivo activity of D2-type MSNs has not been observed in vivo (Calipari et al., 2012). Together remodeling of this circuit in the NAc alters the activity of D1 MSNs, biasing towards D1 signaling pathways and reward-driven behaviors.