Prediction error determines how memories are organized in the brain

Abstract

How is new information organized in memory? According to latent state theories, this is determined by the level of surprise, or prediction error, generated by the new information: a small prediction error leads to the updating of existing memory, large prediction error leads to encoding of a new memory. We tested this idea using a protocol in which rats were first conditioned to fear a stimulus paired with shock. The stimulus was then gradually extinguished by progressively reducing the shock intensity until the stimulus was presented alone. Consistent with latent state theories, this gradual extinction protocol (small prediction errors) was better than standard extinction (large prediction errors) in producing long-term suppression of fear responses, and the benefit of gradual extinction was due to updating of the conditioning memory with information about extinction. Thus, prediction error determines how new information is organized in memory, and latent state theories adequately describe the ways in which this occurs.

Keywords: extinction; fear; latent state; learning; memory; neuroscience; prediction error; rat.

Figure 1.. Cochran and Cisler, 2019 latent state model simulation of standard extinction with spontaneous recovery.

The simulations depict the associative strength (A) and latent state beliefs (B) of a conditioned stimulus (CS) across conditioning, extinction, and spontaneous recovery. Initially, there are 10 conditioning trials (t=1–10, R=1), followed by 80 extinction trials (t=11–90, R=0). A time delay was inserted between trials 90 and 91 to simulate spontaneous recovery before extinction trials continued (t=91–110, R=0). The CS’s associative strength (A) increases across conditioning trials, sharply decreases across extinction trials, and briefly recovers post time delay, indicating a spontaneous recovery of performance. The right panel (B) depicts the degree of belief that the most recent latent state is active. If the belief changes throughout the simulation, this indicates that a new latent state has been inferred. Upon the beginning of extinction (t=11), the figure indicates that a new state has been inferred (due to the large prediction error generated by the absence of the unconditioned stimulus, US). The simulation captures the decrement of performance associated with extinction as well as the recovery of performance that occurs after a time delay.

Figure 2.. Gradual extinction is more effective than standard extinction.

(A) Design for Experiment 1: Rats were conditioned with tone-shock pairings and then either received standard extinction (Group Extinction, n=8, tone alone presentations) or gradual extinction (Group Gradual, n=8, tone-shock pairings – shock intensity decreased across the day). Following, all groups received standard extinction of the tone and were tested for spontaneous recovery two weeks later. Simulation of the latent state model showing the associative strength (B) and latent state beliefs (C) of a gradually extinguished conditioned stimulus (CS) across conditioning, extinction, and spontaneous recovery (Cochran and Cisler, 2019). Initially, there are 10 conditioning trials (t=1–10, R=1), followed by 80 extinction trials (t=11–90). Within the (gradual) extinction trials, the CS is still paired with the unconditioned stimulus (US), however, the intensity of the US is halved every 20 trials (t=11–30, R=0.5; t=31–50, R=0.25; t=51–70, R=0.125) until it is removed for the final 20 extinction trials (t=71–90, R=0). A time delay was inserted between trials 90 and 91, simulating spontaneous recovery, before a final 20 extinction trials (t=91–110, R=0). Associative strength (B) increases across conditioning, decreases steadily across gradual extinction, and remains low after a time delay, indicating an absence of spontaneous recovery. Panel (C) depicts the degree of belief that the most recent latent state is active. If the belief changes throughout the simulation, this indicates that a new latent state has been inferred. The belief remains at 1 across conditioning, extinction, and spontaneous recovery, indicating that the prediction errors produced by gradual extinction are not sufficient to produce state-splitting. Full simulation details are included in Appendix 3. (D) Percentage freezing levels across the final session of extinction (left) and spontaneous recovery (right). Bars represent means ± SEM. Dots represent individual freezing levels (males = dark gray, females = light gray). Freezing levels were similar between groups at the final session of extinction, however, Group Gradual displayed lower freezing than Group Extinction at the spontaneous recovery test. (E) Design for Experiment 2: Rats were conditioned with tone-shock pairings and then either received standard extinction (Group Extinction, n=8), gradual extinction (Group Gradual, n=8), or remained in their home cage (Group Home, n=8). Following, all groups received standard extinction of the tone and were tested for spontaneous recovery and reinstatement. (F) Percentage freezing levels across the final session of extinction (left), spontaneous recovery test (middle), and reinstatement test (right). Bars represent means ± SEM. Dots represent individual freezing levels (males = dark gray, females = light gray). Freezing levels were similar between Groups Gradual and Extinction at the final session of extinction, while Group Home displayed a higher level. Groups Gradual and Extinction displayed less freezing than Group Home at both the spontaneous recovery and reinstatement test. However, while Group Gradual showed less freezing than Group Extinction at both tests, the difference only reached significance at the reinstatement test.

Figure 3.. The effectiveness of gradual extinction is dependent on a progressive reduction in the shock intensity.

(A) Design for Experiment 3: Rats were conditioned with tone-shock pairings and then either received standard extinction (Group Extinction, n=8), gradual extinction (Group Gradual, n=8), or scrambled extinction (Group Scramble, n=8 – same number and intensity of tone-shock pairings as Group Gradual but in a pseudo-random order). Following, all groups received standard extinction of the tone and were tested for spontaneous recovery and reinstatement two weeks later. Simulation of the latent state model showing the associative strength (B) and latent state beliefs (C) of Group Scramble across conditioning, extinction, and spontaneous recovery (Cochran and Cisler, 2019). The trial structure remained identical to the structure used in the simulation displayed in Figure 2C and D with the exception of the 60 (gradual) extinction trials. These trials were the same in number and intensity as Group Gradual (i.e. 20 trials each of R=0.5, R=0.25, and R=0.125) but were now arranged in a pseudo-random order. The order was identical to the order used in the corresponding experiment (see Methods experiment 3 for exact sequence). The standard extinction and spontaneous recovery trials remained the same. Associative strength increases across conditioning, declines across extinction at a slow rate, with a lot of trial-to-trial variability. It declines further across the final extinction trials before recovering at the spontaneous recovery test. Latent state beliefs (C) switched upon the removal of the reward (t=71, R=0) indicating that a new latent state had been inferred. Thus, the simulations show that scrambled extinction is not as effective as gradual extinction in producing robust extinction learning. (D) Percentage freezing levels across the final session of extinction (left), spontaneous recovery test (middle), and reinstatement test (right). Bars represent means ± SEM. Dots represent individual freezing levels (males = dark gray, females = light gray). Freezing levels are similar across the final session of extinction and the spontaneous recovery test for all groups. Group Gradual displayed less freezing than Groups Extinction and Scramble at the reinstatement test. (E) Design for Experiment 4: Rats were conditioned with tone-shock pairings and then either received standard extinction (Group Extinction, n=8), gradual extinction (Group Gradual, n=8) or weak shock extinction. Those who received weak shock extinction were split into two groups who received either 1 or 3 days of pairings (Group 0.1 × 3, n=8 – Three days of tone-shock pairings at the weakest shock intensity; Group 0.1 × 1, n=8 – A single day of tone-shock pairings at the weakest shock intensity and two days of tone-alone presentations). Following, all groups received standard extinction of the tone and were tested for spontaneous recovery and reinstatement two weeks later. Simulation of the latent state model showing the associative strength (F) and latent state beliefs (G) of Group 0.1 × 3 across conditioning, extinction, and spontaneous recovery (Cochran and Cisler, 2019). The trial structure remained identical to the structure used in the simulation displayed in Figure 2C and D with the exception of the 60 gradual extinction trials. All gradual extinction trials now immediately shifted to the lowest intensity reward (t=11–70, R=.125). The remaining extinction and spontaneous recovery trials remained the same. Associative strength (F) increases across conditioning, decreases quickly across the weak shock extinction trials and standard extinction trials before recovering at the spontaneous recovery test. Latent state beliefs (G) change upon the first trial of weak shock extinction, indicating that a new latent state had been inferred. Thus, the simulations show that weak shock extinction is not as effective as gradual extinction in producing robust extinction learning. (H) Percentage freezing levels across the final session of extinction (left), spontaneous recovery test (middle), and reinstatement test (right). Bars represent means ± SEM. Dots represent individual freezing levels (males = dark gray, females = light gray). Freezing levels are similar across the final session of extinction for all groups. Group Gradual displayed less freezing than all other groups at the spontaneous recovery and reinstatement tests, but this difference only reached statistical significance at the spontaneous recovery test.

Figure 4.. The effectiveness of gradual extinction is attenuated by a shift in physical or temporal context.

(A) Design for Experiment 5: Rats were conditioned with tone-shock pairings in context A (Group Extinction, n=8; Group Gradual, n=8) or in context B (Group Extinction-Different, n=8; Group Gradual-Different, n=8). All groups then received extinction in context A, either standard (Groups Extinction and Extinction-Different) or gradual (Groups Gradual and Gradual-Different) extinction. All groups received further standard extinction before being tested for spontaneous recovery (all in context A). Simulation of the latent state model showing the associative strength (B) and latent state beliefs (C) of Group Gradual-Different across conditioning, extinction, and spontaneous recovery (Cochran and Cisler, 2019). The trial structure remained identical to the structure used in the simulation displayed in Figure 2C and D with the exception of a context shift occurring upon the start of gradual extinction (t=11). The standard extinction and spontaneous recovery trials remained the same. Associative strength (B) increases across conditioning, decreases steadily across gradual extinction, and remains low after a time delay. Latent state beliefs (C) remain constant across all trials, indicating that a context shift had not induced state-splitting. Thus, the simulations predict that the effectiveness of gradual extinction is not attenuated by a shift in context. (D) Percentage freezing levels across the final session of extinction (left) and spontaneous recovery test (right). Bars represent means ± SEM. Dots represent individual freezing levels (males = dark gray, females = light gray). Freezing levels are similar across the final session of extinction for all groups. At test, Group Gradual displayed less conditioned stimulus (CS) freezing than Groups Extinction, however, the pattern was reversed for those who received conditioning in a different context (Group Gradual-Different >Group Extinction-Different). (E) Design for Experiment 6: All rats were conditioned with tone-shock pairings and then either received standard extinction or gradual extinction. For half the rats, this occurred on successive days (Group Extinction, n=8; Group Gradual, n=8), while for the other half, these experiences were separated by two weeks (Group Extinction-Delay, n=8; Group Gradual-Delay, n=8). All groups were arranged such that the extinction experiences occurred together. All groups then received further standard extinction and were tested for spontaneous recovery and reinstatement. Simulation of the latent state model showing the associative strength (F) and latent state beliefs (G) of Group Gradual-Delay across conditioning, extinction, and spontaneous recovery (Cochran and Cisler, 2019). The trial structure remained identical to the structure used in the simulation displayed in Figure 2C and D with the exception of a time delay occurring upon the start of gradual extinction (t=11). The standard extinction and spontaneous recovery trials remained the same. Associative strength (F) increased across conditioning, decreased steadily across gradual extinction, and remained low after a time delay. Latent state beliefs (G) remained constant across all trials, indicating that a time delay had not induced state-splitting. Thus, the simulations that the effectiveness of gradual extinction is not attenuated by a time delay between conditioning and extinction. (H) Percentage freezing levels across the final session of extinction (left) and spontaneous recovery test (right). Bars represent means ± SEM. Dots represent individual freezing levels (males = dark gray, females = light gray). CS freezing levels are similar across the final session of extinction for all groups. At the spontaneous recovery and reinstatement test, Group Gradual displayed less CS freezing than Groups Extinction, however, the pattern was reversed for those who received conditioning in a different context (Group Gradual-Delay >Group Extinction-Delay).

Figure 5.. Percentage freezing levels across the final session of extinction and spontaneous recovery test for an omnibus analysis of gradual extinction.

Group Gradual (n=46) contains data from all rats who received a gradual extinction procedure in any experiment, Group Extinction (n=47) contains data from all rats who received standard extinction in any experiment. Data are means ± SEM.

Appendix 1—figure 1.. Percentage conditioned stimulus (CS) freezing (±SEM) across conditioning and extinction for Experiments 1, 2, and 4.

Extinction is split into extinction stage 1 and stage 2. The first stage of extinction refers to extinction sessions where groups were receiving different treatments (i.e. CS alone or continued CS-US pairings). The second stage of extinction is where all groups were receiving CS alone presentations. (A) CS Freezing increased across conditioning trials (left) and decreased across extinction stage 1 (middle) and extinction stage 2 (right) at similar rates for both gradual and standard extinction groups. (B) CS freezing increased across conditioning trials (left) at similar rates for both groups. Group Gradual displayed an increase in freezing across extinction stage 1 while Group Extinction displayed a decrease. Both groups displayed similar levels of CS freezing at extinction stage 2, which appeared to be less than Group No Extinction. (C) CS freezing increased across conditioning trials (left) for all groups, however, Group Gradual appears to have increased less than the other three groups. Group Gradual again shows an increase in freezing across extinction stage 1 (middle) compared to decreases in the remaining three groups. Groups decreased at similar rates across extinction stage 2 (right).

Appendix 1—figure 2.. Percentage conditioned stimulus (CS) freezing (±SEM) across conditioning and extinction for Experiments 5 and 6.

Extinction is split into extinction stage 1 and stage 2. The first stage of extinction refers to extinction sessions where groups were receiving different treatments (i.e. CS alone or continued CS-US pairings). The second stage of extinction is where all groups were receiving CS alone presentations. (A) Freezing increased across conditioning trials (left) at similar rates for all groups. Freezing decreased across extinction trials and at a greater rate for those who received standard extinction compared to gradual extinction (regardless of context) across stages 1 and 2 of extinction. (B) Freezing increased across conditioning (left) and decreased across extinction stages 1 (middle) and 2 (right) at similar rates for all groups.

Appendix 2—figure 1.. Percentage conditioned stimulus (CS) freezing ( ± SEM) across trials for spontaneous recovery and reinstatement tests for Experiments 1–4.

Percentage freezing levels across the spontaneous recovery (A) for experiment 1, spontaneous recovery (B) and reinstatement test (C) for experiment 2, spontaneous recovery (D) and reinstatement test (E) for experiment 3 and spontaneous recovery (F) and reinstatement test (G) for experiment 4.

Appendix 2—figure 2.. Percentage conditioned stimulus (CS) freezing ( ± SEM) across trials for spontaneous recovery and reinstatement tests for Experiments 5–6.

Percentage freezing levels across the spontaneous recovery (A) for experiment 5 and spontaneous recovery (B) and reinstatement test (C) for experiment 6.

Appendix 3—figure 1.. Simulations of the Cochran and Cisler, 2019 latent state model for designs used in Experiments 4–6.

(A). Associative strength (AS) simulation of a conditioned stimulus (CS) that receives extinction and testing for spontaneous recovery. Associative strength increases across conditioning, decreases across extinction before recovering at test, after a time delay, indicating spontaneous recovery. (B). Latent state beliefs (LSB) of a CS that receives extinction and testing for spontaneous recovery. Latent state beliefs switch at the beginning of extinction indicating that a new state has been inferred (due to the large prediction error caused by the absence of the reward). (C). Associative strength simulation of a CS that receives gradual extinction and testing for spontaneous recovery. Associative strength increases across conditioning, decreases steadily across gradual extinction, and remains low after a time delay, indicating an absence of spontaneous recovery. (D). Latent state beliefs of a CS that receives gradual extinction and testing for spontaneous recovery. Latent state beliefs do not change across the simulation, indicating that all stages were encoded into the same state. (E). Associative strength simulation of a CS that receives scrambled extinction and testing for spontaneous recovery. Associative strength increases across conditioning, decreases steadily across scrambled extinction (with high trial-to-trial variability) before recovering briefly after a time delay, indicating spontaneous recovery. (F). Latent state beliefs of a CS that receives scrambled extinction and testing for spontaneous recovery. Latent state beliefs change upon the full removal of the reward in extinction (i.e. after the scrambled stage) indicating state-splitting and the necessity of a progressive reduction in shock intensity to ensure conditioning and extinction are encoded together.

Appendix 3—figure 2.. Simulations of the Cochran and Cisler, 2019 latent state model for designs used in Experiments 4–6.

(A) Associative strength (AS) simulation of a conditioned stimulus (CS) that receives weak shock extinction and testing for spontaneous recovery. Associative strength increases across conditioning, decreases quickly across presentations of the weak shock, and further decreases when the shock is removed before recovering at test, after a time delay, indicating spontaneous recovery. (B) Latent state beliefs (LSB) of a CS that receives weak shock extinction and testing for spontaneous recovery. Latent state beliefs switch at the beginning of weak shock extinction indicating that a new state has been inferred (due to the large prediction error caused by the decrease in shock intensity). (C) Associative strength simulation of a CS that receives gradual extinction in a different context from conditioning and testing for spontaneous recovery. Associative strength increases across conditioning, decreases steadily across gradual extinction, and remains low after a time delay, indicating an absence of spontaneous recovery. (D) Latent state beliefs of a CS that receives gradual extinction in a different context from conditioning and testing for spontaneous recovery. Latent state beliefs do not change across the simulation, indicating that all stages were encoded into the same state despite the physical context shift. (E) Associative strength simulation of a CS that receives gradual extinction after a time delay from conditioning and testing for spontaneous recovery. Associative strength increases across conditioning, decreases steadily across gradual extinction, and remains low after a time delay, indicating an absence of spontaneous recovery. (F) Latent state beliefs of a CS that receives gradual extinction after a time delay from conditioning and testing for spontaneous recovery. Latent state beliefs do not change across the simulation, indicating that all stages were encoded into the same state despite the temporal context shift.