Optimal utility and probability functions for agents with finite computational precision

Abstract

When making economic choices, such as those between goods or gambles, humans act as if their internal representation of the value and probability of a prospect is distorted away from its true value. These distortions give rise to decisions which apparently fail to maximize reward, and preferences that reverse without reason. Why would humans have evolved to encode value and probability in a distorted fashion, in the face of selective pressure for reward-maximizing choices? Here, we show that under the simple assumption that humans make decisions with finite computational precision--in other words, that decisions are irreducibly corrupted by noise--the distortions of value and probability displayed by humans are approximately optimal in that they maximize reward and minimize uncertainty. In two empirical studies, we manipulate factors that change the reward-maximizing form of distortion, and find that in each case, humans adapt optimally to the manipulation. This work suggests an answer to the longstanding question of why humans make "irrational" economic choices.

Keywords: computational precision; prospect theory; uncertainty; utility.

Fig. 1.

Optimal value and probability distortions under decision noise. (A) Optimal (i.e., reward-maximizing) values for $v\left(x\right)$ and $w\left(p\right)$ as derived from Eq. 4 under the assumption that $Y_i=x_i\cdot p_i$ (y axis) plotted against their untransformed counterparts (x axis) under variable levels of decision noise (columns). Note that for convenience the y axis was scaled to unity and its values thus do not reflect supra- or sublinear coding with respect to an ideal observer. (B) Optimal (i.e., reward- and certainty-maximizing) values for $v\left(x\right)$ and $w\left(p\right)$ as derived from Eq. 4 under the assumption that $Y_i=x_i\cdot p_i\cdot \left(1-H_i\right)$ (y axis) plotted against their untransformed counterparts (x axis) under variable levels of decision noise (columns).

Fig. 2.

Illustrating optimal value distortion of parametric utility functions. (A) Each subpanel plots the relative difference in decision utility as a function of the relative expected value of a gamble. Each dot is a unique gamble of the form $\left[x_1,\hspace{0.17em}{p}_1;\hspace{0.17em}{x}_2,\hspace{0.17em}{p}_2\right]$. The x axis denotes $E{V}_1-E{V}_2$ and the y axis denotes $d{v}^{lin}-d{v}^{dis}$, where $dv=U_1-U_2+\epsilon$ under linear $\left(d{v}^{lin}\right)$ and distorted $\left(d{v}^{dis}\right)$ transduction, respectively. For $d{v}^{lin},\hspace{0.17em}\kappa =1,\hspace{0.17em}\gamma =1$ whereas $\kappa /\gamma$ vary from compressive (left columns) to anticompressive (right columns) under increasing levels of noise (top to bottom; $\epsilon \in \left[0.1,\hspace{0.17em}0.3,\hspace{0.17em}0.5\right]$). Red dots signal those gambles where $sign\left(E{V}_1-E{V}_2\right)=sign\left(d{v}^{lin}\right)$ but $sign\left(E{V}_1-E{V}_2\right)\ne sign\left(d{v}^{dis}\right)$ and blue dots signal the converse, i.e., where $sign\left(E{V}_1-E{V}_2\right)\ne sign\left(d{v}^{lin}\right)$ but $sign\left(E{V}_1-E{V}_2\right)=sign\left(d{v}^{dis}\right)$. The number above each plot indicates the relative fraction of blue dots minus red dots; positive numbers thus indicate that there were more gambles where distortion led to more rewarding choices. (B) The relative fraction of decision utilities that were of consistent sign with $E{V}_1-E{V}_2$ under linear recoding (Left), distorted recoding (Middle), and their difference (Right), as a function of noise and distortion level ($\kappa$). The yellow area shows that compression is reward maximizing where noise is nonzero. (C, Left) How the relative density of decision utilities changes with different levels of distortion, under various levels of $\kappa$. (C, Middle) The expected reward obtained, relative to linear transduction, as a function of $E{V}_1-E{V}_2$ under various levels of $\kappa$. (C, Right) The change in probability of choosing gamble 1, relative to linear transduction as a function of $E{V}_1-E{V}_2$, under various levels of $\kappa$.

Fig. 3.

(Upper) Loss landscapes showing parameterizations of the winning model in each group that maximize reward (for the certain outcome condition; A [experiment 1] and B [experiment 2]) and that maximize reward/minimize risk (for the uncertain outcome condition; C [experiment 1] and D [experiment 2]). Warmer colors signal parameterizations that are closer to optimal, and the black circle shows the maximum. Red circles show parameter estimates for individual human participants in each experiment (A and B) Double-exponent model; (C and D) PT model). Note how humans cluster in the same quadrant as the maximum; see statistics below. (Lower) The difference in loss between the distorted model and a linear model, i.e., a model with the same estimated noise but with $\log \left(\kappa \right)=0$ and $\log \left(\gamma \right)=0$. Red circles are individual participants. Warmer colors show regions where the distortion increases return over the linear model. (E) Exceedance probabilities for the PT model and double-exponent model in the certain outcome condition (Left) and uncertain outcome condition (Right).

Fig. 4.

Each panel shows the reward-maximizing form of the functions $v\left(x\right)$ (blue shading) and $w\left(p\right)$ (red shading). Each function is expressed as a density over optimal estimates derived from each participant in the certain outcome condition (A [experiment 1] and B [experiment 2]) and the uncertain outcome condition (C [experiment 1] and D [experiment 2]). Optimal estimates vary from participant to participant because of distinct noise levels and variation in lottery sampling. Superimposed on each reward-maximizing function is the form of the distortion that best fit human choices (estimated from median; parameters, shown on each plot). This was estimated from the double-exponent model for the certain outcome condition (A and B) and from PT for the uncertain outcome condition (C and D).