Biases in preferences for sequences of outcomes in monkeys

Abstract

Movies, vacations, and meals are all examples of events composed of a sequence of smaller events. How do we go from our evaluations of each scene in a movie to an evaluation of the sequence as a whole? In theory, we should simply average the values of the individual events. In practice, however, we are biased towards sequences where each element tends to be better than the previous, where the last value is large, and we overweight the best (or worst) part of the sequence. To study how general these biases are we examined monkeys' preferences for sequences of rewards in a novel reward repeat task. Monkeys were first given a sequence of rewards and then chose between repeating the sequence or receiving a standard comparator sequence. We found that, like humans, monkeys overweight events that happen later in a sequence, so much so that adding a small reward to the end of a sequence can paradoxically reduce its value. Monkeys were also biased towards sequences with large peak values (the highest value in the sequence), but only following a working memory challenge, suggesting that this preference may be driven by memory limitations. These results demonstrate the cross-species nature of biases in preferences for sequences of outcomes. In addition, monkeys' consistent preference for sequences in which large values occur later challenges the generality of discounting models of intertemporal choice in animals.

Keywords: Biases; Heuristics; Intertemporal choice; Macaque; Peak-end.

Fig. 1

Schematic of the design of the reward-repeat task. (A) Stimuli used. On each trial, subjects chose between a probe stimulus (left image) and one of four different standard stimuli (four colored squares, right). Reward units are multiples of 10 μL. (B and C) Timeline of task, two examples. On each trial, subjects were initially presented with a sequence of 5 fluid rewards; each element was separated by 0.5 s; the probe stimulus image appeared centrally on the computer monitor during this time. Then, following a brief delay (0.5 s), the probe stimulus and one standard stimulus appeared on the left and right of the fixation spot (sides were randomized on each trial). (B) Selection of the probe led to a repeat of the probe sequence and (C) selection of the standard led to the appropriate standard sequence. Inter-trial interval was 1 s.

Fig. 2

Basic preference data for all three monkeys for flat sequences. (A) Average proportion of trials on which monkeys chose each of four flat sequences (i.e. sequences with five repeats of the same value). Data are averaged across the four standard stimuli. Monkeys chose larger valued sequences more often, confirming that their behavior was sensible. Error bars indicate standard error. (B) Plot of proportion chosen for the probe sequence [3.2 3.2 3.2 3.2 3.2] as a function of the four different comparator sequences (red dots). Monkeys were more likely to choose higher valued comparators. Black line indicates best-fit sigmoidal curve. Blue line highlights the point of subjective equivalence (PSE), our model estimate of the value at which the monkey was indifferent to the probe and comparator. The PSE provides an estimate of the decision utility assigned to the probe sequence. (C) Plot of the PSE for each of the four flat probe sequences shown in panel A. PSE rose with probe value and was in all cases about ∼0.8 fluid units greater than the comparator value. Error bars indicate standard error and are calculated by a jackknife procedure (see Section 2.3). (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 3

Preferences of three monkeys for ordered sequences. (A) Monkeys assign greater value to increasing sequences than to flat sequences or decreasing sequences, even when total intake is matched. (B) Preference for large value at the end of the sequence is maintained even when small values are zeros (no reward). (C) Monkeys assign greater value to sequences with a single large value the later it is in the sequence. In all cases, error bars indicate standard error as computed by jackknife (see Section 2.3).

Fig. 4

Adding a small reward at the end of a sequence can, paradoxically, reduce its subjective value. This effect does not appear to reflect an attempt to rate maximize, as including a delay (zero) does not significantly reduce preference.

Fig. 5

Impact of position in sequence on upcoming valuation. Results of regression of position against likelihood of choosing in the random sequence variant of the task. Stimuli later in the sequence exhibit a stronger effect on choice. Error bars indicate standard error (see Section 2.3).

Fig. 6

Influence of peak value on preference with and without working memory challenge. (A) Plot of value placed on each of three sequences with same total value but different peak. Subjects prefer sequences with flatter distribution of values. (B) Introducing a weak working memory challenge (a 4 s delay after initial presentation of stimuli and before choice) induces a preference for peaked sequences. Error bars indicate standard error, as calculated by a jackknife procedure (see Section 2.3).