A discounting framework for choice with delayed and probabilistic rewards

Abstract

When choosing between delayed or uncertain outcomes, individuals discount the value of such outcomes on the basis of the expected time to or the likelihood of their occurrence. In an integrative review of the expanding experimental literature on discounting, the authors show that although the same form of hyperbola-like function describes discounting of both delayed and probabilistic outcomes, a variety of recent findings are inconsistent with a single-process account. The authors also review studies that compare discounting in different populations and discuss the theoretical and practical implications of the findings. The present effort illustrates the value of studying choice involving both delayed and probabilistic outcomes within a general discounting framework that uses similar experimental procedures and a common analytical approach.

Figure 1

Choice between a smaller reward, available sooner (SS), and a larger reward, available later (LL). The curved lines represent change in subjective value as a function of time. The heights of the bars represent the actual reward amounts. T₁ = Time 1; T₂ = Time 2.

Figure 2

Experimental procedure in Green et al. (1981). The top diagram represents a trial in which pigeons made a choice at Time 1 (T₁), 28 s before the outcome period, between a smaller, sooner reward and a larger, later reward by pecking one of two keys (indicated by the horizontally and vertically striped circles, respectively). The bottom diagram represents a trial in which pigeons made their choice at Time 2 (T₂), 2 s before the outcome period.

Figure 3

Percentage of choice of the smaller, sooner reward plotted as a function of the time from the choice until the outcome period. Data are reprinted from Behaviour Analysis Letters, 1, L. Green, E. B. Fisher Jr., S. Perlow, and L. Sherman, “Preference Reversal and Self-Control: Choice as a Function of Reward Amount and Delay,” pp. 43–51, 1981, with permission from Elsevier.

Figure 4

Percentage of participants choosing the larger, later (LL) reward plotted as a function of the time until the smaller, sooner (SS) alternative. Data represent results from three conditions, each with a different interval between the SS and the LL reward. Data are from “Temporal Discounting and Preference Reversals in Choice Between Delayed Outcomes,” by L. Green, N. Fristoe, and J. Myerson, 1994, Psychonomic Bulletin & Review, 1, p. 386. Copyright 1994 by the Psychonomic Society. Reprinted with permission.

Figure 5

Procedure for studying temporal discounting. Each of the rectangles represents one of a series of successive choices between a smaller, sooner and a larger, later reward in which the amount of the smaller reward is increased until it is preferred to the larger reward.

Figure 6

The subjective value of a delayed $10,000 reward plotted as a function of the time until its receipt. The curved lines represent alternative forms of the temporal discounting function (Equations 1,2, and 3) fit to the data. Data are from “Discounting of Delayed Rewards: A Life-Span Comparison,” by L. Green, A. F. Fry, and J. Myerson, 1994, Psychological Science, 5, p. 35. Copyright 1994 by Blackwell Publishers, Limited. Reprinted with permission.

Figure 7

The subjective value of a delayed $1,000 reward plotted as a function of the time until its receipt for three age groups (children, young adults, and older adults). The curved lines represent the hyperbola-like discounting function (Equation 3) fit to the data for each group. Data are from Behavioural Processes, 46, L. Green, J. Myerson, and P. Ostaszewski, “Discounting of Delayed Rewards Across the Life Span: Age Differences in Individual Discounting Functions,” pp. 89–96. Copyright 1999 by Elsevier. Printed with permission.

Figure 8

The subjective value of a delayed reward of a year-long vacation plotted as a function of the time until the beginning of the vacation (upper graph), and the subjective value of a delayed reward of 1 year’s free use of a car plotted as a function of the time until use of the car was available (lower graph). For both graphs, subjective value was calculated as a proportion of the amount of money that participants judged equal in value to the reward (vacation or car) when it was available without a delay. The curved lines represent fits of Equations 1, 2, and 3 to the data. Data are from Experiment 4 of “The Effect of Temporal Constraints on the Value of Money and Other Commodities,” by A. Raineri and H. Rachlin, 1993, Journal of Behavioral Decision Making, 6, p. 92. Copyright 1993 by John Wiley & Sons Limited. Reproduced with permission.

Figure 9

Subjective value of delayed real and hypothetical rewards (expressed as a proportion of their nominal value: $10) plotted as a function of the time until their receipt. The curved lines represent fits of Equations 2 and 3 to the combined data from the real and hypothetical rewards. Data are from “Delay Discounting of Real and Hypothetical Rewards,” by G. J. Madden, A. M. Begotka, B. R. Raiff, and L. L. Kastern, 2003, Experimental and Clinical Psychopharmacology, 11, p. 142. Copyright 2003 by the American Psychological Association. Reprinted with permission of the first author.

Figure 10

Procedure for studying probability discounting. Each of the rectangles represents one of a series of successive choices between a smaller, certain, and a larger, probabilistic, reward in which the amount of the smaller reward is increased until it is preferred to the larger reward.

Figure 11

The subjective value of a probabilistic $500 reward plotted as a function of the odds against its receipt. The curved lines represent the expected value and the alternative forms of the probability discounting function (Equations 4 and 5) fit to the data. Data are from Experiment 1 of “Amount of Reward Has Opposite Effects on the Discounting of Delayed and Probabilistic Outcomes,” by L. Green, J. Myerson, and P. Ostaszewski, 1999a, Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, p. 421. Copyright 1999 by the American Psychological Association.

Figure 12

The subjective value of probabilistic $200 and $5,000 rewards (expressed as a proportion of their nominal amounts) plotted as a function of the odds against their receipt (left graph), and the subjective value of delayed $200 and $5,000 rewards as a function of the time until their receipt (right graph). The curved lines represent the hyperbola-like discounting function (Equation 3 in the left graph and Equation 5 in the right graph) fit to the data. Data are from Experiment 2 of “Amount of Reward Has Opposite Effects on the Discounting of Delayed and Probabilistic Outcomes,” by L. Green, J. Myerson, and P. Ostaszewski, 1999a, Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, p. 423. Copyright 1999 by the American Psychological Association.

Figure 13

The subjective value of a delayed reward (expressed as a proportion) plotted as a function of time until its receipt. Data in the left graph are from "Impulsive and Self-Control Choices in Opioid-Dependent Patients and Non-Drug-Using Control Participants: Drug and Monetary Rewards," by G. J. Madden, N. M. Petry, G. J. Badger, and W. K. Bickel, 1997, Experimental and Clinical Psychopharmacology, 5, p. 259. Copyright 1997 by the American Psychological Association. Reprinted with permission of the first author. Data in the right graph are from Figures 1 and 2 of “Impulsivity and Cigarette Smoking: Delay Discounting in Current, Never, and Ex-Smokers," by W. K. Bickel, A. L. Odum, and G. J. Madden, 1999, Psychopharmacology, 146, p. 451. Copyright 1999 by Springer-Verlag. Reprinted with permission. Both studies compared discounting of delayed monetary rewards by substance abusers (opioid-dependent individuals, left graph, and cigarette smokers, right graph) with discounting by controls. In addition, both studies compared discounting of monetary rewards with discounting of the abused substance (heroin, left graph, and cigarettes [cigs], right graph) by members of the substance-abuse group. The curved lines represent the hyperbola-like discounting function (Equation 3) fit to the data.

Figure 14

The subjective value of delayed gains and losses (expressed as a proportion of their nominal amount: $500) plotted as a function of the time until receipt of the reward or payment of the cost. The curved lines represent the hyperbola-like discounting function (Equation 3) fit to the data. Note that although the absolute subjective value of a loss is negative, subjective value expressed as a proportion of the nominal loss is positive because, in this case, subjective value is the ratio of two negatives. Data are from Table 3 of “Delayed Reward and Cost Discounting,” by J. G. Murphy, R. E. Vuchinich, and C. A. Simpson, 2001, Psychological Record, 51, p. 583. Copyright 2001 by Kenyon College. Reprinted with permission.