BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING

Abstract

This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.

Keywords: Pavlovian conditioning; amygdala; extinction; hippocampus; instrumental conditioning; prefrontal cortex; striatum.

Graphical abstract

FIGURE 1.

ABA renewal after the extinction of Pavlovian fear conditioning (top) and appetitive conditioning (bottom). In either case, conditioned stimulus (CS)-unconditioned stimulus (US) pairings occurred in Context A (not shown), extinction occurred in Context A (Ext-A), or Context B (Ext-B) (left), and then testing occurred in Context A (right). Top: ABA renewal in conditioned suppression, the fear conditioning method in which a long-duration (e.g., 60 s) CS is paired with footshock, and the “fear” response is indexed by the CS suppression of a baseline of lever-pressing for food; y-axis: the suppression ratio, the number of lever press responses during the CS divided by the number of responses in the CS plus the number in a similar period before the CS. Lower scores indicate more conditioned fear. Extinction (left) occurred at the same rate regardless of whether it occurred in the conditioning context (A) or the other context (B). Renewal in Group Ext-B was strong (right), but fear did not recover to the level of that in a group that received no extinction (NE). [From Bouton and King (19).] Bottom: ABA renewal in appetitive conditioning. Here the behavioral measure is the extent to which an auditory CS associated with a food-pellet US evoked head-jerking behavior. Extinction (left) again occurred at the same rate regardless of context. Extinction is more context-specific than conditioning [From Bouton and Peck (20)]. [Figure reprinted from Bouton (14) with permission from the publisher.]

FIGURE 2.

A cartoon depicting what is learned in Pavlovian extinction. During extinction, a new inhibitory association is learned between the conditioned stimulus (CS) and unconditioned stimulus (US) (blocked red line). The excitatory CS-US association learned during conditioning (green arrow) remains intact; it can activate the representation of the US, causing conditioned responding (not shown). However, its effect is canceled by activation of the inhibitory CS-US association. The inhibitory association is activated by the extinction context (green arrow). Outside the extinction context, the inhibitory association is not enabled, and response recovery occurs. The extinction context thus controls responding by selecting the CS’s inhibitory association with the US.

FIGURE 3.

ABA renewal of responding in an operant conditioning setting. Rats learned to press an “active” lever for an intravenous mix of heroin and cocaine. Extinction: A: responding on the active lever was then extinguished over daily extinction sessions. Inset: responding in groups that received extinction in the same context or a different context from the one in which they had lever pressed for drug. B: responding on the inactive lever. Test: active and inactive lever responding in a Control group tested in the extinction context; a Novel group that was tested in a new context; and a Renewal group that was tested in the original acquisition context after extinction in a different context. Note that responding in this group was at a level comparable to level at the start of extinction. [From Crombag and Shaham (59).]

FIGURE 4.

Associative structures that could in principle control responding during ABA renewal in Pavlovian conditioning (A and B) and instrumental learning (C and D). In either case, the extinction Context B could conceivably inhibit performance through any of the associations shown, and the conditioning Context A could activate performance through any of the associations shown (see text for details).

FIGURE 5.

Neural circuits mediating Pavlovian fear conditioning, extinction retrieval, and renewal. Left: auditory conditioned stimuli (CS) are processed by the thalamic medial geniculate nucleus (MG) and auditory cortex (AC) en route to the basolateral nucleus of the amygdala (BLA). This information is associated with the footshock unconditioned stimulus (US), which is conveyed by the parabrachial (PB) nucleus and other relays, in the amygdala. Contextual information is processed in the hippocampus (HPC) and medial prefrontal cortex, which are interconnected by the nucleus reuniens (RE). HPC projections and MG nucleus projections to the BLA are involved in the expression of fear conditioned responses (CRs) to auditory and contextual stimuli, respectively. Projections from the central nucleus of the amygdala (CeA) to the ventrolateral midbrain periaqueductal gray (PAG) are involved in the freezing CR. The dorsolateral PAG mediates activity burst unconditioned responses (UR) to the shock US. Middle: the retrieval of extinction memories is associated with infralimbic (IL) cortical inhibition of BLA neurons involved in CR production to auditory and contextual stimuli. IL projections to the RE may suppress retrieval of contextual fear memories. Right: presentation of an extinguished CS outside the extinction context causes a return in the freezing CR, and this is mediated by HPC-mediated excitation of the fear CR, as well as inhibition of IL neurons involved in suppression of the CR.

FIGURE 6.

Synaptic plasticity models for fear conditioning and extinction. Left: long-term potentiation (LTP) in axons conveying conditioned stimulus (CS) information to the basolateral amygdala (BLA) principal neurons (PN) results in the expression of a fear conditioned response. Right: extinction may be supported by several different synaptic changes including depotentiation of the LTP induced during conditioning, recruitment of LTP at CS afferents on amygdala inhibitory interneurons (IN), or LTP at inhibitory interneuron synapses on amygdala PNs (LTPi). All three forms of plasticity would be expected to reduce conditional responding, but only LTP (interneuron) and inhibitory LTP would allow for recovery of the fear CR after extinction.

FIGURE 7.

Neural circuits mediating extinction, extinction retrieval, and renewal in instrumental learning. Confirmed causal connectivity is indicated by unbroken lines, unconfirmed connectivity by broken lines. The symmetrical inhibitory (left) and excitatory (right) influences of context depend, at least in part, on neuronal ensembles distributed across the prelimbic (PL) and infralimbic (IL) prefrontal cortex. During extinction learning and extinction retrieval, these ensembles converge with glutamatergic inputs from basolateral amygdala (BLA), ventral hippocampus (HPC), and paraventricular thalamus (PVT) to interface with D1 receptor expressing spiny projection neurons in the nucleus accumbens shell (AcbSh) that preferentially provide inhibitory GABAergic synaptic input to lateral hypothalamic (LH) GABA neurons. During ABA renewal, neuronal ensembles distributed across PL and IL converge with glutamatergic inputs from ventral HPC and PVT to interface with D1 receptor expressing spiny projection neurons in the AcbSh that preferentially project to ventral tegmental area (VTA) [directly and indirectly via ventral pallidum (VP)] and LH. VP also projects to subthalamic nucleus (STN) and VTA. Lateral VTA dopamine neurons mediate renewal via an ascending dopaminergic signal to the striatum and possibly cortex. LH is the only known site of convergence between the extinction and renewal circuits, with the same LH GABAergic neurons receiving monosynaptic inhibitory GABAergic input from both extinction (AcbSh) and renewal (VP) circuits. OFC, orbitofrontal cortex; AcbC, nucleus accumbens core.

FIGURE 8.

A: parallel and partially segregated cortical-striatal thalamic-cortical circuits allow different forms of behavioral control during instrumental learning. Contextual control over instrumental behavior after extinction is embedded in a “selection” loop involving AcbSh. Reinstatement of extinguished instrumental behavior is embedded in an ‘invigoration’ loop involving AcbC. Separate loops involving dorsal striatum determine goal-directed action (dorsomedial striatum [DMS]) and habitual (dorsolateral striatum [DLS]) response control. AcbSh, nucleus accumbens shell; AcbC, nucleus accumbens core; IL, infralimbic cortex; LH, lateral hypothalamus; MD, mediodorsal thalamus; PL, prelimbic cortex; PO, posterior thalamus; PVT, paraventricular thalamus; SM, somatosensory cortex; SN, substantia nigra; VTA, ventral tegmental area. B: within the “selection loop,” AcbSh D1→LH projection mediates response inhibition in extinction (rose color), whereas AcbSh D1→VPm→LH/VTA mediates response restoration during renewal (green color). The VPm→VTA pathway emerges from VP parvalbumin (PV) neurons whereas the VP→LH pathway emerges from VP Gad1 neurons. The extinction and renewal pathways converge onto the same LH Gad1 neurons. These same selection circuits interface with hypothalamic and midbrain behavioral control columns to mediate the response-generative effects of nonreinforcement.