Neural correlates of variations in event processing during learning in basolateral amygdala

Abstract

The discovery that dopamine neurons signal errors in reward prediction has demonstrated that concepts empirically derived from the study of animal behavior can be used to understand the neural implementation of reward learning. Yet the learning theory models linked to phasic dopamine activity treat attention to events such as cues and rewards as static quantities; other models, such as Pearce-Hall, propose that learning might be influenced by variations in processing of these events. A key feature of these accounts is that event processing is modulated by unsigned rather than signed reward prediction errors. Here we tested whether neural activity in rat basolateral amygdala conforms to this pattern by recording single units in a behavioral task in which rewards were unexpectedly delivered or omitted. We report that neural activity at the time of reward is providing an unsigned error signal with characteristics consistent with those postulated by these models. This neural signal increased immediately after a change in reward, and stronger firing was evident whether the value of the reward increased or decreased. Further, as predicted by these models, the change in firing developed over several trials as expectations for reward were repeatedly violated. This neural signal was correlated with faster orienting to predictive cues after changes in reward, and abolition of the signal by inactivation of basolateral amygdala disrupted this change in orienting and retarded learning in response to changes in reward. These results suggest that basolateral amygdala serves a critical function in attention for learning.

Figure 1.

Task, behavior, and recording sites. a, Line deflections indicate the time course of stimuli (odors and rewards) presented to the animal on each trial. Other trial events are listed below. At the start of each recording session, one well was randomly designated as short (a 0.5 s delay before reward) and the other long (a 1–7 s delay before reward) (block 1). In the second block of trials these contingencies were switched (block 2). In blocks 3–4, we held the delay constant while manipulating the number of the rewards delivered. b, The impact of delay length and reward size choice behavior on free-choice trials. Bar graphs show average percentage choice for short versus long or big versus small across all free-choice trials. c, The impact of delay length and reward size on behavior on forced-choice trials. Bar graphs show percentage correct (left) and reaction time (right) across all recording sessions for different delays (top) and sizes (bottom). d, Location of recording sites. Gray dots represent final electrode position. Gray box marks extent of recording sites. Asterisks indicate planned comparisons revealing statistically significant differences (t test, p < 0.05). CeA, Central nucleus of amygdala; LaA, lateral amygdala; ABL, basolateral amygdala. Error bars indicate SEMs.

Figure 2.

Neural activity in ABL is increased in response to unexpected reward delivery and omission. a, Heat plots showing average activity over all ABL neurons (n = 58) that showed a significant effect of delay or size in an ANOVA during the 1 s after reward delivery. Activity over the course of the trials is plotted during the first and last 20 (10 per direction) trials in each training block (Fig. 1 a; blocks 1–4). Activity is shown, aligned on odor onset (“align odor”) and reward delivery (“align reward”). Blocks 1–4 are shown in the order performed (top to bottom). Thus, during block 1, rats responded after a “long” delay or a “short” delay to receive reward (actual starting direction—left/right—was counterbalanced in each block and is collapsed here). In block 2, the locations of the “short” delay and “long” delay were reversed. In blocks 3–4, delays were held constant but the size of the reward (“big” or “small”) varied. Line display between heat plots shows the rats' behavior on free-choice trials that were interleaved within the forced-choice trials. Value of 50% means that rats responded the same to both wells. b, Distribution of indices [(early − late)/(early + late)] representing the difference in firing to reward delivery (1 s) and omission (1 s) during trials 3–10 (early) and during the last 10 trials (late) after upshifts (i) (2^sh, 3^bg, and 4^bg) and downshifts (ii) (2^lo and 4^sm). Note the first 2 trials were excluded in calculating the contrasts based on data in Figure 3 b, showing that firing did not increase on these initial trials, consistent with predictions of the Pearce–Hall model. Filled bars in distribution plots (i, ii) indicate the number of cells that showed a main effect (p < 0.05) of learning (early vs late). iii, Correlation between contrast indices shown in i and ii. Filled points in scatter plot (iii) indicate the number of cells that showed a main effect (p < 0.05) of learning (early vs late). Black diamonds indicate those neurons that also showed an interaction with shift type (upshift vs downshift). Analysis and figures shown here include both free- and forced-choice trials. p values for distributions are derived from Wilcoxon test.

Figure 3.

Neural activity in ABL increases gradually in response to unexpected reward and omission consistent with Pearce–Hall attention. a, c, Signals predicted by the Pearce–Hall (a) and Rescorla–Wagner (c) models after unexpected delivery (black) and omission (gray) of reward. b, d, Average firing (500 ms after reward delivery) in ABL (b) and for dopamine (d) neurons in VTA during the first 10 trials in blocks 2^sh, 3^bg, and 4^bg (upshifts = black) and in blocks 2^lo and 4^sm (downshifts = gray) normalized to the maximum. As reported previously (Roesch et al., 2007), dopamine neurons exhibit very short phasic changes in firing to unexpected reward and omission. The same time epoch was used to analyze both datasets here, to make the analysis equivalent. Asterisks indicate a significant difference between the first and third trial. See Results for ANOVA. Analysis and figures shown here include both free- and forced-choice trials. Error bars indicate SEMs.

Figure 4.

Activity in ABL was correlated with odor-port orienting. a, Speed at which rats initiated trials after house light illumination during the first and last 10 trials in blocks 2–4. These data were normalized to the maximum and inverted for ease of comparison with the model and neural data in Figure 3. b, Correlation between changes in firing in ABL on trial n and the orienting response on trial n + 2. Analysis and figures shown here include both free- and forced-choice trials. Error bars indicate SEMs.

Figure 5.

Inactivation of ABL disrupted odor-port orienting and impaired learning in the task. a, Speed at which rats oriented to the odor port after house-light illumination. Shown is the average latency in the first two trials after a change in reward versus the last two trials before a change in reward for vehicle versus NBQX sessions. For this comparison, the very first trial in a block was counted as the last trial before a change in reward, since orienting to the odor port on that trial preceded knowledge of the change in reward value. b, Choice performance in vehicle versus NBQX sessions, plotted according to whether the well values in a particular trial block were similar to or opposite from those learned at the end of the prior session. c, The height of the bars represents a perseveration score in vehicle versus NBQX sessions, computed by averaging choice performance in “similar” trial blocks with 100 minus choice performance in “opposite” blocks. See Results for statistics. Error bars indicate SEMs.