A runs-test algorithm: contingent reinforcement and response run structures

Abstract

Four rats' choices between two levers were differentially reinforced using a runs-test algorithm. On each trial, a runs-test score was calculated based on the last 20 choices. In Experiment 1, the onset of stimulus lights cued when the runs score was smaller than criterion. Following cuing, the correct choice was occasionally reinforced with food, and the incorrect choice resulted in a blackout. Results indicated that this contingency reduced sequential dependencies among successive choice responses. With one exception, subjects' choice rule was well described as biased coin flipping. In Experiment 2, cuing was removed and the reinforcement criterion was changed to a percentile score based on the last 20 reinforced responses. The results replicated those of Experiment 1 in successfully eliminating first-order dependencies in all subjects. For 2 subjects, choice allocation was approximately consistent with nonbiased coin flipping. These results suggest that sequential dependencies may be a function of reinforcement contingency.

Keywords: behavioral variability; lever press; mutual uncertainty; rat; sequential dependency; the runs test.

Fig 1

All possible scores on the runs test in the number of sample  =  20, calculated from Equation 3. Response proportion is n_x / (n_x + n_y), in which n_x is the less-chosen response alternative. White cells signify data within the +/− 1.96 criterial range, while grey cells fall outside this range.

Fig 2

Proportions of absolute values of S in three ranges: smaller than 1, between 1 and 2, and larger than 2, in each session of Experiment 1. In Area A, the criterial range for S scores was +/− 1.96, and there were additional food deliveries when the stimulus lights were off. In Area B, there was no additional food. In Area C, the criterial range was reduced to +/− 1.39.

Fig 3

Proportions of responses that illuminated the cue lights in each session of Experiment 1. Areas A, B, and C are the same as for Figure 2.

Fig 4

Mutual uncertainties in Experiment 1. The difference (T_m) between successive uncertainty indices (H_m and H_m+1) for each subject for each order of sequential dependency. Horizontal lines indicate critical values for chi square. Data points below the critical value represent no significant difference between H_m and H_m+1. See text for calculations of Hs, Ts and transformations to chi square.

Fig 5

Conditional probability profiles of right responses for the first seven sessions with the criterion set at +/− 1.96 and the last seven sessions with it set at +/− 1.39, in Experiment 1. Each set of seven connected points, Lags 0 to 6, correspond to one session. The horizontal dotted line represents p of .5. The first point of each profile is the relative frequency of right responses (R/(R+L)). The next four points are the conditional probabilities at each lag.

Fig 6

The relative distributions of four-response units in the first session of Experiment 1, the last session with the criterion set at +/− 1.96, and the last session with it set at +/− 1.39, arranged in successive columns. Lines are expected values from randomness.

Fig 7

Proportions of absolute values of S in three ranges: smaller than 1, between 1 and 2, and larger than 2 in each session of Experiment 2. The area before the vertical line is baseline and that after it is the runs-test phase.

Fig 8

Mutual uncertainties of last five sessions under each phase of Experiment 2. Horizontal lines indicate critical values for chi square. Data points below the critical value represent no significant difference between H_m and H_m+1. See text for calculations of Hs, Ts and transformations to chi square.

Fig 9

Conditional probability profiles of right responses for the last five sessions in the baseline and runs test phases of Experiment 2. Each set of seven connected points, Lags 0 to 6, correspond to one session. The horizontal dotted line represents p of .5. The first point of each profile is the relative frequency of right responses (R/(R+L)). The next four points are the conditional probabilities at each lag.

Fig 10

The relative distributions of four-response units. The left column is for the first session of Experiment 2, the middle column is for the last session of baseline, and the right column is for the last session of the runs-test phase. Lines are expected values from randomness.