Habituation of reinforcer effectiveness

Abstract

In this paper we propose an integrative model of habituation of reinforcer effectiveness (HRE) that links behavioral- and neural-based explanations of reinforcement. We argue that HRE is a fundamental property of reinforcing stimuli. Most reinforcement models implicitly suggest that the effectiveness of a reinforcer is stable across repeated presentations. In contrast, an HRE approach predicts decreased effectiveness due to repeated presentation. We argue that repeated presentation of reinforcing stimuli decreases their effectiveness and that these decreases are described by the behavioral characteristics of habituation (McSweeney and Murphy, 2009; Rankin etal., 2009). We describe a neural model that postulates a positive association between dopamine neurotransmission and HRE. We present evidence that stimulant drugs, which artificially increase dopamine neurotransmission, disrupt (slow) normally occurring HRE and also provide evidence that stimulant drugs have differential effects on operant responding maintained by reinforcers with rapid vs. slow HRE rates. We hypothesize that abnormal HRE due to genetic and/or environmental factors may underlie some behavioral disorders. For example, recent research indicates that slow-HRE is predictive of obesity. In contrast ADHD may reflect "accelerated-HRE." Consideration of HRE is important for the development of effective reinforcement-based treatments. Finally, we point out that most of the reinforcing stimuli that regulate daily behavior are non-consumable environmental/social reinforcers which have rapid-HRE. The almost exclusive use of consumable reinforcers with slow-HRE in pre-clinical studies with animals may have caused the importance of HRE to be overlooked. Further study of reinforcing stimuli with rapid-HRE is needed in order to understand how habituation and reinforcement interact and regulate behavior.

Keywords: ADHD; behavioral regulation; dopamine; drug addiction; obesity; operant conditioning; psychomotor stimulant; sensory reinforcement.

FIGURE 1

In this experiment, the sensory reinforcer of light-onset was contingent on snout-poking. Light was presented according to one of three schedules (FR 1, VI 1 min, and VI 6 min; data for VI 1 min not shown). The experiment had two phases, one in which animals were pre-exposed to an experimental chamber for 10 sessions, followed by the second phase of 10 sessions in which animals could snout-poke into one of two holes to produce 5 s light-onset. The data are depicted as two session blocks and show the rate of responding in 6 min epochs of each 30 min test session. BL 1 indicates responding at the end of the pre-exposure phase when there was no response-contingent light-onset. Tests 1–5 indicate two session blocks of light-contingent responding.

FIGURE 2

Demonstration of stimulus specificity (A) and dishabituation (B). Rats were trained to respond for response-contingent light-onset presented in the front of the test chamber according to a VI 1 min schedule of reinforcement. Stimulus specificity (A) was tested by shifting the location of the response-contingent light to the rear of the test chamber during the last 30 min of the test session [indicated by slanted lines in (A)]. Dishabituation (B) was tested by turning on a loud warbling tone during minutes 30–36 of the test session [indicated by slanted lines in (B)]. The data are plotted in 10 6-min epochs for the 60 min test sessions. Baseline (BL) responding during each epoch is plotted as a percent of total responding during the first epoch. Test-day responding was divided by the number of responses during BL for each epoch.

FIGURE 3

Calculation of habituation rate (HR). (A) Data having different absolute rates of responding plotted as five equal duration epochs. (B) The same data with each epoch plotted as a percentage of total responding. The lines with arrows in plot (B) indicate the difference between the first epoch of the test session and the epoch with the lowest percentage of responding. This difference is divided by the time between the first epoch and the epoch with the lowest percentage of responding to produce the HR measure shown in plot (C). See text for details.

FIGURE 4

Nicotine [NIC, (A)] and methamphetamine [METH, (B)] decrease the HRE of sensory reinforcers. The plots in the left column show responding for a sensory reinforcer (5 s light-onset) after administration of NIC [(A); saline and 0.4 mg/kg] and METH [(B); saline, 0.25, and 1.0 mg/kg]. NIC data shows a 40 min session consisting of five 8-min epochs. METH data shows a 50 min session consisting of five 10-min epochs. The average number of responses per epoch is labeled as data point “Avg.” Both left column plots show that when treated with saline, responding systematically decreased, indicating HRE. Asterisks (*) indicate a within-group difference in responding between saline and drug (p < 0.05). Administration of NIC (A) and METH (B) both slowed HRE. METH increased overall responding but NIC did not. The histogram bars on the right show habituation rate (HR) (see text for explanation of HR).

FIGURE 5

The effect of 0.5 mg/kg methamphetamine (METH) on choice between a sensory reinforcer and water in water-restricted rats. The asterisk (*) indicates a difference in responding between saline and METH treated rats (p < 0.05) Rats treated with saline responded for the sensory reinforcer <15% of the time. Rats treated with METH responded for the sensory reinforcer about 30% of the time.

FIGURE 6

Within-session analysis of the effect of methamphetamine on concurrent schedule performance for water and sensory reinforcers. The left plot shows within-session changes in responding for a sensory reinforcer presented according to a VI 1 min schedule in rats treated with METH (0.5 mg/kg) and saline. Asterisks (*) in the left plot indicate differences in responding (p < 0.05) The middle plot shows within-session responding for a small (0.025 mL) water reinforcer concurrently presented with the visual stimulus shown in the left plot. The histogram plot on the right shows HRE for sensory and water reinforcers and the effects of saline and METH treatment on HRE. In the histogram, asterisks (*) indicate an overall difference in Habituation Rate between VS and water reinforcers (p < 0.05), and a difference in Habituation Rate between the saline and METH treated rats receiving the VS reinforcer (p <0.05).

FIGURE 7

Operant responses emitted by the organism produce sensory consequences. If the response-contingent sensory stimulus is unexpected, it increases DA neurotransmission. DA neurotransmission increases the probability that the animal will repeat responses that preceded the onset of the sensory stimulus. Reinforcing effectiveness is operationally defined as response rate. The comparator process depicted in the cartoon determines the novelty of the sensory consequence. Past occurrences of the sensory consequence result in an inhibitory input into the comparator that cancels the effects of the sensory consequence DA neurotransmission. The canceling signal is the integral of the number of response-contingent sensory stimulus presentations so that the strength of the canceling signal is a function of the number of previous sensory reinforcer presentations. The cartoon shows that 10 repetitions caused less intense inhibition than 30 repetitions, which in turn caused less inhibition than 100 repetitions. The conceptual model shows how DA neurotransmission decreases as a function of reinforcer repetition and how changes in DA neurotransmission are hypothesized to underlie HRE.