Neural Basis of Strategic Decision Making

Abstract

Human choice behaviors during social interactions often deviate from the predictions of game theory. This might arise partly from the limitations in the cognitive abilities necessary for recursive reasoning about the behaviors of others. In addition, during iterative social interactions, choices might change dynamically as knowledge about the intentions of others and estimates for choice outcomes are incrementally updated via reinforcement learning. Some of the brain circuits utilized during social decision making might be general-purpose and contribute to isomorphic individual and social decision making. By contrast, regions in the medial prefrontal cortex (mPFC) and temporal parietal junction (TPJ) might be recruited for cognitive processes unique to social decision making.

Keywords: arbitration; game theory; prefrontal cortex; reinforcement learning.

Figure 1. Activity in the medial prefrontal cortex related to learning and simulating the actions of others

A. Activity in the dorsomedial prefrontal cortex correlated with the simulated other’s action prediction errors (top) and activity in the ventromedial prefrontal cortex correlated with the simulated reward prediction error (bottom) [36]. B. Spatial gradient in the activity within a set of regions of interests (top) related to the value of chosen option for self vs. others in prosocial (middle) and selfish (bottom) individuals [62].

Figure 2. Neuronal activity in the dorsomedial prefrontal cortex related to strategic decision making

A. Monkeys deviated systematically from model-free reinforcement learning to avoid the exploitation by the computer opponent during a biased matching pennies task, in which the payoff to the animal was determined jointly by the choices of the monkey and computer (left). S and R, safe and risky target, respectively, and the number in the payoff matrix indicates the number of tokens received by the animal. The color of each box in the decision tree (middle) and the position of each circle in the scatter plot (right) indicate how much the probability of choosing the safe target following a particular choice-outcome history deviated from a model-free reinforcement learning algorithm, P (safe)-PRL(safe), and how this increased or decreased the probability of winning compared to the equilibrium strategy, P(token)-P_NASH(token). R−, R+, denote loss and gain from the risky target, whereas S0 and S+ indicate neutral outcome and gain from the safe target. B. The strength of neural signals (ordinate) related to the tendency to switch away from a model-free reinforcement learning (abscissa) across different choice-outcome history (shown in A) for a single neuron in the dorsomedial prefrontal cortex. C. Same neural-behavioral correlation for switching for the population of neurons recorded in the dorsolateral prefrontal cortex (dlPFC), dorsomedial prefrontal cortex (dmPFC), dorsal anterior cingulate cortex (ACCd), lateral intraparietal area (LIP), caudate nucleus (CN), and ventral striatum (VS). White and gray bars indicate the results obtained from a token-based biased matching pennies task [45, 86] and a symmetric matching pennies task [44, 48], respectively.