Decision making, movement planning and statistical decision theory

Abstract

We discuss behavioral studies directed at understanding how probability information is represented in motor and economic tasks. By formulating the behavioral tasks in the language of statistical decision theory, we can compare performance in equivalent tasks in different domains. Subjects in traditional economic decision-making tasks often misrepresent the probability of rare events and typically fail to maximize expected gain. By contrast, subjects in mathematically equivalent movement tasks often choose movement strategies that come close to maximizing expected gain. We discuss the implications of these different outcomes, noting the evident differences between the source of uncertainty and how information about uncertainty is acquired in motor and economic tasks.

Figure 1

a) An example of a stimulus configuration presented on a display screen. The subject must rapidly reach out and touch the screen. If the screen is hit within the green circle, 2.5 cents is awarded. If within the red, there is a penalty of 12.5 cents. The circles are small (9 mm radius) and the subject can’t completely control this rapid movement. In Box 1 we explain what the subject should do to maximize winnings. b) A comparison of subjects’ performance to the performance that would maximize expected gain. The shift of subjects’ mean movement end points from the center of the green target region is plotted as a function of the shift of mean movement end point that would maximize expected gain for 5 different subjects (indicated by 5 different symbols) and the 6 different target-penalty configurations shown in Fig. 2a (replotted from [6], Figure 5a).

Figure 1

a) An example of a stimulus configuration presented on a display screen. The subject must rapidly reach out and touch the screen. If the screen is hit within the green circle, 2.5 cents is awarded. If within the red, there is a penalty of 12.5 cents. The circles are small (9 mm radius) and the subject can’t completely control this rapid movement. In Box 1 we explain what the subject should do to maximize winnings. b) A comparison of subjects’ performance to the performance that would maximize expected gain. The shift of subjects’ mean movement end points from the center of the green target region is plotted as a function of the shift of mean movement end point that would maximize expected gain for 5 different subjects (indicated by 5 different symbols) and the 6 different target-penalty configurations shown in Fig. 2a (replotted from [6], Figure 5a).

Figure 2

a) Trial-by-trial deviation of movement end point (in the horizontal direction) from the mean movement end point in that condition as a function of trial number after introduction of rewards and penalties (reward: 2.5 cents; penalty −12.5 cents); the six different lines correspond to the six different spatial conditions of target and penalty offset as shown on the right (data replotted from [6], Figure 7). b) Trend of a hypothetical simple learning model in which a subject changes motor strategy gradually in response to rewards and penalties incurred. The subject initially aims at the center of the green circle. Before the subject’s first trial in the decision-making phase of the experiment, the subject is instructed that red circles carry penalties and green circles carry rewards. Subjects may approach the aim point maximizing expected gain by slowly shifting the aim point away from the center of the green circle until the winnings match the maximum expected gain. However, the data shown in a) do not support this learning model.

Figure 2

a) Trial-by-trial deviation of movement end point (in the horizontal direction) from the mean movement end point in that condition as a function of trial number after introduction of rewards and penalties (reward: 2.5 cents; penalty −12.5 cents); the six different lines correspond to the six different spatial conditions of target and penalty offset as shown on the right (data replotted from [6], Figure 7). b) Trend of a hypothetical simple learning model in which a subject changes motor strategy gradually in response to rewards and penalties incurred. The subject initially aims at the center of the green circle. Before the subject’s first trial in the decision-making phase of the experiment, the subject is instructed that red circles carry penalties and green circles carry rewards. Subjects may approach the aim point maximizing expected gain by slowly shifting the aim point away from the center of the green circle until the winnings match the maximum expected gain. However, the data shown in a) do not support this learning model.

Box 1’s Figure 1

Equivalence of a movement task and decision making under risk. Subjects must touch a computer screen within a short period of time (e.g. 700 msec). Subjects can win 2.5 cents points by hitting inside the green circle, lose 12.5 cents by hitting inside the red circle, lose 10 cents by hitting where green and red circle overlap or win nothing by hitting outside the stimulus configuration. Each possible aim point on the computer screen corresponds to a lottery. a) Expected gain for a subject aiming at the center of the green target (aim point indicated by the diamond). Black points indicate simulated end points for a representative subject (with 5.6 mm end point standard deviation); target and penalty circles have radii of 9 mm. This motor strategy yields an expected loss of 2.8 cents/trial. The numbers shown below the target configuration describe the lottery corresponding to this aim point, i.e., the probabilities for hitting inside each region and the associated gain. b) Expected gain for a subject with the same motor uncertainty as in a). Here, we simulate the same subject aiming towards the right of the target center (the diamond) to avoid accidental hits inside the penalty circle. This strategy results in an expected gain of 0.78 cents/trial and corresponds to the strategy (aim point) maximizing expected gain. c) Each possible aim point corresponds to a lottery and has a corresponding expected gain, shown by the grayscale background with four particular aim points highlighted.

Box 1’s Figure 1

Equivalence of a movement task and decision making under risk. Subjects must touch a computer screen within a short period of time (e.g. 700 msec). Subjects can win 2.5 cents points by hitting inside the green circle, lose 12.5 cents by hitting inside the red circle, lose 10 cents by hitting where green and red circle overlap or win nothing by hitting outside the stimulus configuration. Each possible aim point on the computer screen corresponds to a lottery. a) Expected gain for a subject aiming at the center of the green target (aim point indicated by the diamond). Black points indicate simulated end points for a representative subject (with 5.6 mm end point standard deviation); target and penalty circles have radii of 9 mm. This motor strategy yields an expected loss of 2.8 cents/trial. The numbers shown below the target configuration describe the lottery corresponding to this aim point, i.e., the probabilities for hitting inside each region and the associated gain. b) Expected gain for a subject with the same motor uncertainty as in a). Here, we simulate the same subject aiming towards the right of the target center (the diamond) to avoid accidental hits inside the penalty circle. This strategy results in an expected gain of 0.78 cents/trial and corresponds to the strategy (aim point) maximizing expected gain. c) Each possible aim point corresponds to a lottery and has a corresponding expected gain, shown by the grayscale background with four particular aim points highlighted.

Box 2’s Figure 1. Motor decisions from experience

In the learning phase of the experiment subjects learn to hit targets. Their performance improves until their movement variability has reached a plateau. During training they have the opportunity to learn their own motor uncertainty but nothing about the training task requires that they do so. In the experimental phase subjects plan movements that trade off the risk of incurring penalties against the possible reward of hitting targets. They show little evidence of learning and perform well in the task. This suggests that they can convert what they learned in the training phase into the information needed to plan effective movements under risk: the equivalent of estimating the probabilities of the various outcomes associated with any proposed aim point, followed by a computation of expected gain.

Box 3’s Figure 1

Example of applying statistical decision theory to modeling goal-directed movement under visual and motor uncertainty. a) A dinner guest, whose arm is shown, intends to pick up the salt shaker at the center of the table with his right hand. An intended trajectory is shown along with a “confidence interval” to indicate the range of other trajectories that might occur. b) The actual executed movement may deviate from the intended and, instead of grasping the salt shaker, the guest may accidentally knock over his full wine glass. c) If executed successfully, the dinner guest will pick up the salt shaker without experiencing social disaster. (Drawings by Andreas Olsson)

Box 3’s Figure 2

Application of statistical decision theory to complex visuo-motor tasks. The goal is a mapping from sensory input V to a movement plan s(V). Gains and losses g,(t,w) are determined by the actual trajectory t executed in the actual state of the world w. The movement plan that maximizes expected gain depends on both visual uncertainty and motor uncertainty. (Here, we follow the convention that random variables are in upper case, e.g. X, while the corresponding specific values that those variables can take on are in lower-case, e.g. p(X).)