Neurocomputations of strategic behavior: From iterated to novel interactions

Abstract

Strategic interactions, where an individual's payoff depends on the decisions of multiple intelligent agents, are ubiquitous among social animals. They span a variety of important social behaviors such as competition, cooperation, coordination, and communication, and often involve complex, intertwining cognitive operations ranging from basic reward processing to higher-order mentalization. Here, we review the progress and challenges in probing the neural and cognitive mechanisms of strategic behavior of interacting individuals, drawing an analogy to recent developments in studies of reward-seeking behavior, in particular, how research focuses in the field of strategic behavior have been expanded from adaptive behavior based on trial-and-error to flexible decisions based on limited prior experience. We highlight two important research questions in the field of strategic behavior: (i) How does the brain exploit past experience for learning to behave strategically? and (ii) How does the brain decide what to do in novel strategic situations in the absence of direct experience? For the former, we discuss the utility of learning models that have effectively connected various types of neural data with strategic learning behavior and helped elucidate the interplay among multiple learning processes. For the latter, we review the recent evidence and propose a neural generative mechanism by which the brain makes novel strategic choices through simulating others' goal-directed actions according to rational or bounded-rational principles obtained through indirect social knowledge. This article is categorized under: Economics > Interactive Decision-Making Psychology > Reasoning and Decision Making Neuroscience > Cognition.

Keywords: decision neuroscience; game theory; social cognition; strategic behavior.

FIGURE 1

Illustration of strategic learning algorithms. To provide a better comparison against other learning rules, only one of the belief learning strategies (i.e., action‐based learning) is illustrated here, although belief learning can be achieved by differential algorithmic processes. Mental‐state learning involves learning about various mental states, including but not limited to the “states” of others' preferences (Ferreira et al., 2021), pro‐sociality tendency (Ray et al., 2008), and the level of sophistication in strategic reasoning (Yoshida et al., 2008). The dashed lines reflect potential relationships that may or may not exist depending on the specific interactive scenario

FIGURE 2

(a) Putative generative models in visual perception and social communication. In visual perception, the brain infers sensory causes (e.g., the apple in sight) from the bodily effects (e.g., the retinal image) by modeling the sensation‐generating process and then inverting this model to derive the most probable cause of the sensation. For social communication, a listener decodes the hidden intention from a received expression by internally simulating the speaker's choice process. (b) Referential communication game involves random‐matched pairs of a speaker and a listener. The speaker's goal is to refer to one of three objects presented in a context, by selecting an expression denoting either the color or the shape feature of the target. The listener, who faces the same context but does not know the target, is required to recover the target according to the expression received from the speaker. The computational model (Frank & Goodman, ; Franke & Degen, 2016) proposes an inverse inferential process, by which a listener simulates the intention‐action contingency from the speaker's perspective and then inverts this process to identify the most probable target. (c) Listener's internal generative model predicts that, for best helping the audience recognize the intended meaning, speakers would compare between candidate expressions and select the expression that conveys the maximal amount of information in context. (d) Listener's vmPFC encodes the intention (e.g., referring to the blue circle)—action (e.g., selecting the expression “blue”) contingencies of the speaker in a manner consistent with the model prediction, even in situations where such generative signals are not necessary for recovering the intention (e.g., three cases shown in the right panel). Adapted from Mi et al. (2021). vmPFC, ventromedial prefrontal cortex