A Free-Operant Reward-Tracking Paradigm to Study Neural Mechanisms and Neurochemical Modulation of Adaptive Behavior in Rats

Abstract

The ability to respond flexibly to changing environmental circumstances is a hallmark of goal-directed behavior, and compromised flexibility is associated with a wide range of psychiatric conditions in humans, such as addiction and stress-related disorders. To identify neural circuits and transmitter systems implicated in the provision of cognitive flexibility, suitable animal paradigms are needed. Ideally, such models should be easy to implement, allow for rapid task acquisition, provide multiple behavioral readouts, and permit combination with physiological and pharmacological testing and manipulation. Here, we describe a paradigm meeting these requirements and employ it to investigate the neural substrates and neurochemical modulation of adaptive behavior. Water-restricted rats learned to emit operant responses for positive reinforcement (water reward) within minutes in a free-operant conditioning environment. Without further training, animals were able to track changes in the reward schedule. Given prior evidence that the medial prefrontal cortex (mPFC) and the dopaminergic system are required for flexible behavior, we aimed to assess both in more detail. Silencing of mPFC compromised flexible behavior when avoidance of punishment was required. Systemic injections of the D2-receptor agonist quinpirole and the D2-receptor antagonist eticlopride had complex, differential impacts on reward seeking and adaptive behavior.

Keywords: dopamine receptors; matching law; muscimol; operant conditioning; punishment; reversal learning.

Figure 1

Picture of the operant chamber with a rat poking into the center (reward) port. Reward (30 µL of water) could be triggered by poking into the left (L) and right (R) ports at random intervals. Water reward is delivered at the center (C) port. The stainless steel grid floor was used to deliver mild foot shocks in later experiments.

Figure 2

Rats pick up the basic task structure in the very first training session. (A) Cumulative numbers of operant responses in 1-min bins (time intervals) for pokes into the left (blue), right (red), and center (reward, yellow) port for one example animal on the first day of training. (B) Cumulative responses into the center port within 2 s after left (blue) or right pokes (red) for the same animal. (C) Cumulative numbers of reinforcers (in bins of 10 min) for all six rats across the first three sessions (separated by vertical dotted lines). The color code key for panels CDEF is given at the bottom of the figure. (D) The number of reinforcers obtained per 10-min bin over the first three days, normalized to the maximum number of reinforcers obtained in any bin by a given rat. For five out of six rats, the maximum number was obtained well before the end of the third session. (E) Relative response proportions P(R_L) for the six rats over the first three days. (F) As in E, but after subtraction of programmed relative reinforcement proportions P(Rf_L) from P(R_L) to highlight that all animals attained matching (horizontal line, zero difference) by the third day of training.

Figure 3

Rats quickly adapt when reinforcement contingencies change every three days. (A) Adaptation to new reward contingencies over five blocks. The panel depicts P(R_L) of an example animal over time. Horizontal dotted lines represent P(R_L) which would perfectly match P(Rf_L) within each block. Vertical lines separate blocks of conditions. (B) Matching behavior after three days of constant contingencies for all three animals. Regression lines were fitted to the data separately for each animal using Equation (2). Dotted black main diagonal represents the strict matching relation.

Figure 4

Rats quickly adapt when reinforcement contingencies change multiple times per session. (A) Dynamic response allocation resulting from changes in P(Rf_L) across all 15 experimental sessions in 10-min bins for an example animal. Blue and red symbols denote change-points for responses (asterisks) and reinforcements (squares). Thin horizontal dotted line denotes unbiased responding; vertical dotted lines denote changes in P(Rf_L). (B) Relative choice proportion before and after changes in reinforcement contingency (vertical dotted line) for infrequent (purple), moderately frequent (green), and frequent (cyan) changes. The data from the other rats tested with different sequences were highly similar. (C) Comparison of relative response proportions during the last five minutes prior to transitions across all three conditions.

Figure 5

Adaptation to punishment and involvement of mPFC. (A) Mean P(R_L) in consecutive, non-overlapping 2-min bins across animals after saline (blue) or muscimol (red) infusion. P(R_L) was rectified, such that values <0.5 signify a preference for the unpunished response port. Under saline, the animals exhibited a clear preference for the non-punished option. Vertical dotted line highlights the time of punishment introduction. (B) As in (A), but showing operant responses. (C) As in (A), but showing the number of rewards retrieved per time bin. (D) As in (A), but showing the number of non-retrieved rewards in all panels. Shading represents SEM.

Figure 6

Systemic administration of dopamine D2-receptor compounds alters task performance. (A) Left: Quinpirole administration reduced overall responding to both response ports (L and R), but not at the reward port (C). Right: Total number of pokes at the L, R and C ports displayed separately for saline (Sal) and quinpirole (Qnp). (B) Quinpirole significantly increased the number of non-retrieved rewards. (C) A total number of triggered rewards with those that were retrieved in blue and those that were not retrieved in black. (D) P(R_L) before (Block 1) and after (Block 2) mid-session change in the reward schedule, which required the animals to adapt their responses to the reversed reward ratio as in the first half of the session. Animals changed their responses, according to the reinforcement ratio (indicated in the graph) after saline administration, but not after quinpirole administration. Dashed horizontal lines indicate optimal behavior as predicted by the Matching law. (E–G) As in A–D, but for behavior after eticlopride vs. vehicle administration. In all box plots, individual data points (black dots) are laid over a box depicting the 25th and 75th percentiles; the horizontal red mark indicates the median, whiskers extend to the most extreme observations, outliers are plotted individually. Asterisks indicate significant differences between drug and vehicle as determined by paired samples t-tests (** ≤ 0.01, *** ≤ 0.001, **** ≤ 0.0001).

Figure 7

Bilateral microinfusion sites in the mPFC. (A) Cannula locations in mPFC, colored dots represent cannula tips of individual rats. Brain diagrams adapted from Reference [66]. (B) Coronal slice of the right hemisphere with Evans Blue fluorescence region (right) and schematic representation of the guide cannula (left). The fluorescence is restricted to the prelimbic portion of mPFC. (C) Image of a coronal slice under bright field illumination. (D) Magnified image of the Evans Blue spread shown in (B).