Complex dynamic processes in sign tracking with an omission contingency (negative automaintenance)

Abstract

Hungry pigeons received food periodically, signaled by the onset of a keylight. Key pecks aborted the feeding. Subjects responded for thousands of trials, despite the contingent nonreinforcement, with varying probability as the intertrial interval was varied. Hazard functions showed the dominant tendency to be perseveration in responding and not responding. Once perseveration was accounted for, a linear operator model of associative conditioning further improved predictions. Response rates during trials were correlated with the prior probabilities of a response. Rescaled range analyses showed that the behavioral trajectories were a kind of fractional Brownian motion.

Figure 1

The probability of recording a peck over blocks of 20 trials for the 4 pigeons in Experiment 1. Intertrial intervals (ITI) are designated at the top of each graph.

Figure 2

The probability of recording a peck over blocks of 20 trials for the 4 pigeons in Experiment 2. ITI = intertrial interval; VI = variable interval.

Figure 3

The probability of a peck on a trial conditional on the events (peck, P, or quiet, Q) on prior trials. The open circles over 1 show the probability that a single trial with a Q will be followed by a P, the open circles over 2 give the probability that a sequence of two Qs will end with a P, and so on. This curve is the hazard function for Ps. The filled circles over 1 show the probability that a sequence of Ps will last for only one trial, the filled circles over 2 show the probability that it will last for only two trials, and so on. This curve is the complement of the hazard function for Qs. The database for each point decreased geometrically with n; graphing ceased when a condition contained fewer than 15 observations. The last panel shows the hazard functions for simulated data from the perseveration model diagrammed in Figure 4.

Figure 4

Top panel: A linear operator model of perseveration. With each trial with a peck (peck state, signified by a left circle) the probability (p) of a peck on the next trial increases by the fraction γ of the distance to 1.0. On each trial without a peck (quiet state, signified by a right circle) the probability of observing a peck on the next trial decreases by the fraction γ of the distance to 0. This constitutes an exponentially weighted moving average of the probability of responding. Bottom panel: A linear operator model of learning to associate the keylight with food and thus approach it. The rate of learning the association is given by the parameter α. On trials without a response, the probability of a response on the next trial increases by the fraction α of the distance to 1.0. On trials with a response there is no food, and the probability decreases by the fraction α of the distance toward 0. In the two-parameter learning model, it decreases by the fraction β, where β is the rate of extinguishing the association.

Figure 5

The average probability of a key peck for all pigeons conditional on the events—a Q followed by a P (filled circles) or a P followed by a Q (open circles)—on the two prior trials. Data plotted at 1, giving the probability of a response on the trials after the doublets, differ by 3.3 points. Q = quiet; P = peck.

Figure 6

The response rate of all subjects in Experiment 2 on all trials is plotted as a function of the probability of a response predicted by an exponentially weighted moving average of the probabilities on the preceding trials. The parameter of the moving average was γ from the first column of Table 1. The curves are the predicted loci given a constant probability of responding through all of the trial (solid line) or during the postlatency epoch (dashed line).

Figure 7

Filled circles: The log10 likelihood ratio (LLR) of the perseveration model with respect to the predictions afforded by the data from Pigeon 5, as the parameter γ is swept over its range. Squares: the LLR of the learning model in tandem with the perseveration model as α is swept over its range with γ held at its (new) optimal value. The higher maximum indicates a better account of the data. Diamonds: the LLR of the two-parameter learning model as β is swept over its range, with the other parameters held at their optimal values. The marginal advantage of each additional model is the difference in their ordinates at their peaks. These differences are displayed in Table 1.

Figure 8

Top left panel: 2,048 uniformly distributed integer random variables. Bottom left panel: the Hurst diagram for the variates above it. Top right panel: the variates in the first panel are summed and detrended to generate a random walk around X_R = 0. Bottom right panel: the Hurst diagram for the variates above it. The construction of Hurst diagrams is detailed in the text and Appendix.

Figure 9

Hurst diagrams, in which the rescaled range is plotted as a function of the epoch over which it is calculated, from Experiment 1. Theoretical slopes of these functions can range from 0 through 0.50 (random variates) to 1.00 (random increments, or Brownian motion).

Figure 10

Hurst diagrams from Experiment 2.

Figure 11

Hurst diagrams for simulated data. The parameters give the values of γ and α.