Rats that sign-track are resistant to Pavlovian but not instrumental extinction

Abstract

Individuals vary in the extent to which they attribute incentive salience to a discrete cue (conditioned stimulus; CS) that predicts reward delivery (unconditioned stimulus; US), which results in some individuals approaching and interacting with the CS (sign-trackers; STs) more than others (goal-trackers; GTs). Here we asked how periods of non-reinforcement influence conditioned responding in STs vs. GTs, in both Pavlovian and instrumental tasks. After classifying rats as STs or GTs by pairing a retractable lever (the CS) with the delivery of a food pellet (US), we introduced periods of non-reinforcement, first by simply withholding the US (i.e., extinction training; experiment 1), then by signaling alternating periods of reward (R) and non-reward (NR) within the same session (experiments 2 and 3). We also examined how alternating R and NR periods influenced instrumental responding for food (experiment 4). STs and GTs did not differ in their ability to discriminate between R and NR periods in the instrumental task. However, in Pavlovian settings STs and GTs responded to periods of non-reward very differently. Relative to STs, GTs very rapidly modified their behavior in response to periods of non-reward, showing much faster extinction and better and faster discrimination between R and NR conditions. These results highlight differences between Pavlovian and instrumental extinction learning, and suggest that if a Pavlovian CS is strongly attributed with incentive salience, as in STs, it may continue to bias attention toward it, and to facilitate persistent and relatively inflexible responding, even when it is no longer followed by reward.

Keywords: Discriminative stimuli; Extinction; Goal tracking; Incentive motivation; Occasion setting; Pavlovian conditioning; Sign tracking.

Figure 1

Acquisition of Pavlovian conditioned approach behavior in STs (n = 72) and GTs (n = 61) from experiments 1–4. Sign-tracking and goal-tracking behaviors during the CS period are shown in six measures (mean ± SEM): A) the probability of deflecting the lever, B) the probability of entering the food cup, C) the number of lever deflections, D) the number of food cup head entries, E) the latency to deflect the lever, F) and the latency to enter the food cup.

Figure 2

In experiment 1, GTs (n = 21) showed faster extinction of PCA behavior than STs (n = 28). The figures show the mean (+ SEM) probability of food cup approach in GTs (A) and lever approach in STs (B) during extinction sessions compared to the last PCA session (baseline). Top panels show within-session probability of approach in 3-trial blocks. A) GTs showed significant decreases in approach during the first extinction session (Ext 1) and the fourth extinction session (Ext 4) compared to baseline. B) STs continued to approach the lever throughout the first extinction session (Ext 1), but showed a significant decrease in approach by Ext 4. When behavior was averaged across all 25 trials per session (C), and the first three trials of each session (D), the GTs showed significant decreases from baseline in earlier sessions than the STs; * p< .05, *** p< .001, change relative to baseline.

Figure 3

In experiment 2, rewarded (R) and non-rewarded (NR) periods were indicated by a change in chamber illumination (red or white lights illuminated for the duration of 15-trial blocks). The figure shows the mean (+ SEM) probability of approaching the lever or food cup in STs (n = 7) and GTs (n = 11), averaged in 3-trial blocks during R and NR periods on days 1 and 5 of discrimination training. The dashed horizontal lines near the top of each panel show mean probability of lever approach (in STs) or food cup approach (in GTs) recorded during the final day of PCA training (PCA 5), prior to discrimination training. GTs showed rapid discrimination between R and NR phases in the very first session (A), and some improvement by the fifth session (B). In contrast, STs showed no discrimination in session 1 (C), but by the fifth session STs discriminated between R and NR periods (D). Note that although in this and subsequent figures the R block is shown as occurring first, in fact, for half the animals the NR block occurred first, and the order was also counter-balanced across days. There was no effect of whether the R or NR blocks occurred first in the session.

Figure 4

In experiment 2, GTs (n = 11) learned to discriminate between R and NR periods faster than STs (n = 7). Behavioral responses (mean + SEM) during R versus NR periods, as well as the last day of PCA training (PCA 5), are shown for lever-directed behavior in STs (A–C) and food cup-directed behavior in GTs (D–F). The STs gradually learned to discriminate between R and NR periods with repeated training, and showed significant differences by session 5. The GTs showed rapid discrimination that was significant in all 5 sessions; **p< .01, ***p< .001, post hoc t tests comparing R and NR blocks. G–I) Difference scores (R responses - NR responses) were significantly higher in GTs than STs; ++ p< .01, +++ p< .001, significant group effects; †p< .05, ††p< .01, post hoc t tests comparing STs and GTs.

Figure 5

In experiment 3, R and NR periods were signaled by spatially and temporally discrete cues (a cue light located at the front or back of the chamber illuminated for only 6s prior to the CS). The figure shows the within-session probability of GTs (n = 18) and STs (n = 15) approaching the lever or food cup (mean + SEM). The dashed horizontal lines near the top of each panel show mean probability of lever approach (for STs) and food cup approach (for GTs) during the last day of PCA training. GTs showed excellent discrimination in both session 1 (A) and session 5 (B), whereas STs showed no discrimination in session 1 (C) and improvement by session 5 (D).

Figure 6

In experiment 3, as in Experiment 2, GTs (n = 18) showed faster discrimination between R and NR periods than STs (n = 15). The figures show mean (+ SEM) lever-directed behavior in STs (A–C) and food cup-directed behavior in GTs (D–F) during the last PCA training day (PCA 5) and then the 5 discrimination sessions. STs initially showed no discrimination between R and NR periods, but significant differences emerged by the third session. In contrast, the GTs showed significant discrimination in all 5 sessions; *p< .05, **p< .01, ***p< .001, post hoc t tests comparing R and NR blocks. G–I) For probability, contacts, and latency, differences between R and NR blocks were significantly greater in GTs than STs; +++p< .001, significant group effects.

Figure 7

In experiments 2 and 3, we examined behavior during the first trial of R and NR blocks, when the light signals indicated whether or not the following CS would be reinforced. Responding during the very first trial of a new block was compared to the average of the preceding 3 trials in the previous block, and difference scores were calculated (R minus NR, black symbols labeled R), and after the switch from rewarded to non-rewarded phases (NR minus R, white symbols labeled NR). In experiment 2 both STs (A) and GTs (B) learned to discriminate between the diffuse light cues in later sessions, showing an increase in contacts during the first trial of R phases and a decrease in contacts during the first trial of NR phases. In experiment 3, STs learned to discriminate between the discrete light cues by the last session (C), but GTs did not (D); * p< .05, significant interaction effects.

Figure 8

In experiment 4 STs (n = 14) and GTs (n = 10) were first trained to make an instrumental response (nose poke) for a food reward. A) STs and GTs initially acquired stable responding for food pellets at similar rates. Panel A shows active and inactive nose-poke (NP) responses during 8 days of acquisition training, broken into 3 phases with different reinforcement schedules and session lengths (FR1, fixed ratio 1; VI, variable interval). B) After training, when R and NR periods were introduced, both groups showed significant discrimination in all 10 sessions. However, the GTs showed higher rates of responding during R blocks than STs; *p < .05, main effect of group. C) Difference scores calculated from active nose-pokes (R minus NR) were also significantly higher in GTs than STs; *p < .05, main effect of group. However, the absence of an interaction effect indicates the two groups learned the discrimination at the same rate. All results are mean ± SEM.