Rethinking reinforcement: allocation, induction, and contingency

Abstract

The concept of reinforcement is at least incomplete and almost certainly incorrect. An alternative way of organizing our understanding of behavior may be built around three concepts: allocation, induction, and correlation. Allocation is the measure of behavior and captures the centrality of choice: All behavior entails choice and consists of choice. Allocation changes as a result of induction and correlation. The term induction covers phenomena such as adjunctive, interim, and terminal behavior-behavior induced in a situation by occurrence of food or another Phylogenetically Important Event (PIE) in that situation. Induction resembles stimulus control in that no one-to-one relation exists between induced behavior and the inducing event. If one allowed that some stimulus control were the result of phylogeny, then induction and stimulus control would be identical, and a PIE would resemble a discriminative stimulus. Much evidence supports the idea that a PIE induces all PIE-related activities. Research also supports the idea that stimuli correlated with PIEs become PIE-related conditional inducers. Contingencies create correlations between "operant" activity (e.g., lever pressing) and PIEs (e.g., food). Once an activity has become PIE-related, the PIE induces it along with other PIE-related activities. Contingencies also constrain possible performances. These constraints specify feedback functions, which explain phenomena such as the higher response rates on ratio schedules in comparison with interval schedules. Allocations that include a lot of operant activity are "selected" only in the sense that they generate more frequent occurrence of the PIE within the constraints of the situation; contingency and induction do the "selecting."

Keywords: Phylogenetically Important Event; allocation; contingency; correlation; inducer; inducing stimulus; induction; reinforcement; reinstatement.

Fig 1

Behavior induced by periodic food. Left: Activities of a pigeon presented with food every 12 s. Right: Activities of a rat presented with food every 30 s. Activities that disappeared before food delivery could not be reinforced. Reprinted from Staddon (1977).

Fig 2

A hypothetical illustration of the concept of allocation. The times spent in various activities in a typical day of a typical student add up to 16 h, the other 8 being spent in sleep. If more time is spent in one activity, less time must be spent in others, and vice versa.

Fig 3

A hypothetical example illustrating change of behavior due to induction as change of allocation. At Time 1, a pigeon's activities in an experimental space exhibit Allocation 1 in response to periodic food deliveries. At Time 2, the interval between food deliveries has been lengthened, resulting in an increase in aggression toward a mirror. Other activities decrease necessarily.

Fig 4

A hypothetical example illustrating change of allocation due to correlation. At Time 1, a child's classroom activities exhibit Allocation 1, including high frequencies of disruptive activities. At Time 2, after the teacher's attention has been made contingent upon being on task, Allocation 2 includes more time spent on task and, necessarily, less time spent in disruptive activities.

Fig 5

Diagram illustrating that Phylogenetically Important Events (PIEs) constitute a subset of PIE-related events, which in turn are a subset of all events.

Fig 6

Results of an experiment in which food itself served as a discriminative stimulus. The number of responses made during an hour of extinction following 40 response-produced food deliveries decreased across the 10 sessions of the experiment. These responses would be analogous to “errors.” The data were published in a paper by Bullock and Smith (1953).

Fig 7

Cartoon of Reid's (1958) results with reinstatement of responding following extinction upon a single response-independent presentation of the originally contingent event (food for rats and pigeons; token for students). A burst of responding (circled) followed the presentation, suggesting that the contingent event functioned as a discriminative stimulus or an inducer.

Fig 8

Cumulative record that Skinner (1948) presented as evidence of “reconditioning” of a response of hopping from side to side. Food was delivered once a minute independently of the pigeon's behavior. Food deliveries (labeled “reinforcements”) are indicated by arrows. No response immediately preceded the first and possibly the second food deliveries (circled), indicating that the result is actually an example of reinstatement. Copyright ©1948 by the American Psychological Association. Reprinted by permission.

Fig 9

Why a contingency or correlation is not simply a temporal relation. The 2-by-2 table shows the conjunctions possible of the presence and absence of two events, E1 and E2. A positive contingency holds between E1 and E2 only if two conjunctions occur with high probability at different times: the presence of both and the absence of both (indicated by checks). The conjunction of the two alone (contiguity) cannot suffice.

Fig 10

Different correlations or contingencies induce either the target activity or other-than-target activities, depending on whether the Phylogenetically Important Event (PIE) involved usually enhances or reduces fitness (reproductive success) by its presence.

Fig 11

Cumulative record of a squirrel monkey pressing a lever that produced electric shock at the end of the fixed interval. Even though the monkey had previously been trained to avoid the very same shock, it now continues to press when pressing produces the shock. This seemingly paradoxical result is explained by the molar view of behavior as an example of induction. Reprinted from Malagodi, Gardner, Ward, and Magyar (1981).

Fig 12

How contingency completes a loop in which an operant activity (B) produces an inducing event (S), which in turn induces more of the activity. O stands for organism. E stands for environment. Left: induction alone. Right: the contingency closes the loop. Induction occurs because the operant activity is or becomes related to the inducer (S; a PIE).

Fig 13

Results from an experiment showing discrimination of correlation. Four cumulative records of complete sessions from 4 pigeons are shown. At the beginning of the session, the correlation between pecking and feeding is positive; if a peck occurred anywhere in the scheduled time interval, food was delivered at the end of the interval. Following the first vertical line, the correlation switched to negative; a peck during the interval canceled food at the end of the interval. Peck rate fell. Following the second vertical line, the correlation reverted to positive, and peck rate rose again. Adapted from Baum (1981a).

Fig 14

Results from an experiment in which lever pressing undergoing extinction was enhanced by presentation of a tone that had previously been paired with the food that had been used in training the lever pressing. Because the tone and lever had never occurred together before, the molecular view had difficulty explaining the effect of the tone on the pressing, but the molar view explains it as induction of pressing as a food-related activity. The data were published by Estes (1948).

Fig 15

Effects of contingency. In a baseline condition with no contingency, little of the to-be-operant activity (e.g., lever pressing or wheel running; the to-be-induced activity) occurs, while a lot of the to-be-contingent PIE-related activity (e.g., eating or drinking; the inducing activity) occurs. This is the baseline allocation (upper left point and broken lines). After the PIE (food or water) is made contingent on the operant activity, whatever allocation occurs must lie on the solid line. The arrow points to the new allocation. Typically, the new allocation includes a large increase in the operant activity over baseline. Thus, contingency has two effects: a) constraining possible allocations; and b) making the operant behavior PIE-related so it is induced in a large quantity.

Fig 16

Example of a feedback function for a variable-interval schedule. The upper curve, the feedback function, passes through the origin and approaches an asymptote (60 PIEs per h; a VI 60s). Its equation appears at the lower right. The average interval t equals 1.0. The parameter a equals 6.0. Response rate is expected to vary from time to time, as shown by the distribution below the feedback function (frequency is represented on the right-hand vertical axis).

Fig 17

Example of an empirical feedback function for a variable-interval schedule. Successive 120-s time windows were evaluated for number of lever presses and number of food deliveries. The food rate and press rate were calculated for each number of presses per 120 s. The unfilled diamonds show the food rates. The feedback equation from Figure 15 was fitted to these points (t  =  0.52; a  =  0.945). The frequency distribution (right-hand vertical axis) below shows the percent frequencies of the various response rates. The filled square shows the average response rate.