The expected value of control: an integrative theory of anterior cingulate cortex function

Abstract

The dorsal anterior cingulate cortex (dACC) has a near-ubiquitous presence in the neuroscience of cognitive control. It has been implicated in a diversity of functions, from reward processing and performance monitoring to the execution of control and action selection. Here, we propose that this diversity can be understood in terms of a single underlying function: allocation of control based on an evaluation of the expected value of control (EVC). We present a normative model of EVC that integrates three critical factors: the expected payoff from a controlled process, the amount of control that must be invested to achieve that payoff, and the cost in terms of cognitive effort. We propose that dACC integrates this information, using it to determine whether, where and how much control to allocate. We then consider how the EVC model can explain the diverse array of findings concerning dACC function.

Figure 1. Anatomy and connectivity of the dorsal anterior cingulate cortex (dACC)

A-B) Cytoarchitectonic subdivisions of human (A) and macaque (B) medial prefrontal cortices. The cingulate sulcus (cgs) has been opened up in both. dACC typically refers to areas 24a-d and the dorsal extent of area 32 (32’ in A, 32(s) in B). Panel A focuses specifically on paracingulate regions of the medial surface, and the color-coding reflects Vogt et al.’s (2004) four-region model. The region referred to as human dACC throughout the main text is the anterior portion of mid-cingulate cortex (aMCC), encompassing an area Picard and Strick (1996) referred to as the rostral cingulate zone (RCZ).. C) Cortical projections to regions of dACC (left; areas 24a-b in yellow, areas 24c-d in orange) and more posterior regions of dorsomedial PFC (right; supplementary and primary motor cortices in pink and purple, respectively) in the macaque (cf. panel B). Relative to the more posterior regions, projections to dACC are much more widespread and include regions of orbital and rostrolateral PFC, temporal and parietal cortices, and insula. D-E) Patterns of resting-state functional connectivity estimated in human (D) and macaque (E) brains using fMRI. The colors in panel D label seven networks within which activity between the regions (of a given color) is highly correlated at rest. Under this parcellation scheme, regions of dACC span the frontoparietal network (orange; often referred to as the “control” network) and the ventral attention network (violet). Panel E shows a parcellation of resting-state connectivity networks focused on the connectivity of cingulate cortex regions-of-interest with the rest of the brain. Patterns of connectivity for the “executive” network (shown in red) and the “attention-orienting” network (dark blue), particularly within lateral PFC, suggest potential homologues with human frontoparietal and ventral attention networks. However, exact boundaries and homologies between dACC across species remain ambiguous (see, e.g., Cole et al., 2009). Panel A reprinted from Palomero-Gallagher et al. (2009) by permission of John Wiley and Sons; panels B-C reprinted from Morecraft et al. (2012), © (2012), modified with permission from Elsevier; panel D cortical and striatal connectivity reprinted from Yeo et al. (2011) and Choi et al. (2012), respectively, with permission from The American Physiological Society; panel E reprinted from Hutchison et al. (2011), by permission of Oxford University Press.

Figure 2. The Expected Value of Control (EVC) model applied to the Stroop task

A) A model of the Stroop task illustrating the three major components of cognitive control: monitoring, specification and regulation. Thickness of the connections and size of the units denote the amplitude of the signal along each processing pathway. The figure also illustrates how the model can be extended to include conflict monitoring that, in turn, can be used to specify the strength of the control signal needed to support processing in the task-relevant pathway. Note that the model ascribes to dACC roles in monitoring and specification, and to the lPFC a role in regulation. B) The EVC model. Control signal specification involves choosing a control signal that maximizes EVC. For illustration, we diagram here the processes underlying specification for the Stroop color-naming task (illustrated in panel A). The objective is to select a control signal that maximizes the EVC. This, in turn, requires comparison of signals that differ in their identity (here, word-reading vs. color-naming task sets) and intensity (represented in the figure by meters). In both cases, the EVC estimation takes into account both the costs of each candidate signal and its expected payoffs and expected outcomes (arrow weights indicate transition probabilities). Although identity and intensity are segregated here for the purposes of illustration, they are fully integrated in the EVC estimation as specified in Eq. 1, and likely reflect the operation of a common set of mechanisms (see Section 5.3). Moreover, unlike specifying the intensity of a given control signal (right), the process of specifying control signal identities (left) does not require mutual exclusivity (i.e., multiple control signal identity-intensity pairings can be simultaneously specified as a single array).

Figure 3. The relationship of conflict adaptation effects in human dACC to control monitoring versus specification

A) Measuring from single units in human dACC, Sheth et al. (2012) found a parametric effect of current trial conflict (example neuron shown), an effect that has been widely reported in neuroimaging studies (see Section 4.1). Left and right sides of this figure plot firing rate changes aligned to stimulus and response onsets, respectively. Note that this effect alone can be indicative of either monitoring of demands and/or specification of different intensities of control accordingly. B) Left: This group also found evidence of conflict adaptation (Gratton et al., 1992), with high-conflict (incongruent) trials requiring greater control, and therefore exhibiting longer RTs, when following a low-conflict (congruent) trial (cI) than when following another incongruent trial (iI). Right: This behavioral effect was abolished after these individuals underwent cingulotomy. C) Previous fMRI studies have tied sequential adjustment effects to a particular pattern of responses in dACC: greater activity on cI than iI trials. This pattern has been observed in numerous experiments using different tasks and manipulations, including Botvinick et al., 1999 (left), Carter et al., 2000 (center), and Kerns et al., 2004 (right). It has been interpreted as reflecting a monitoring function, since greater dACC activity was observed under conditions of high conflict but low control. D) Strikingly, single unit recording data from Sheth et al. show the opposite pattern, with higher firing rates on iI than cI trials, a pattern consistent with control-signal specification. The presence of both monitoring and specification signals in dACC is consistent with the EVC theory. Determining why one function manifests in fMRI and the other in single-unit recording presents an important challenge for further research, and the EVC model may be of use in guiding such investigations. Panels A, B, and D reprinted by permission from Macmillan Publishers Ltd: Nature, Sheth et al. (2012), © (2012). Panel C: [left] reprinted by permission from Macmillan Publishers Ltd: Nature, Botvinick et al. (1999), © (1999); [center] from Carter et al. (2000), © (2000) National Academy of Sciences, USA; [right] from Kerns et al. (2004), with permission from AAAS.

Figure 4. The influence of incentives and task difficulty on control allocation and dACC activity

The EVC model predicts shifts in control intensity in response to changes in task incentives (panel A) and in task difficulty (B). In each case, control intensity is specified based on a maximization of the EVC (blue curves). As indicated in Eq.s 1 and 2, the EVC depends, in turn, on both the expected payoffs and costs for candidate control signals (see also Figure 2B). Payoffs (green curves) can vary with signal intensity due to resulting changes in task performance. For example, a stronger control signal might yield more accurate performance, and therefore greater payoffs. However, the inherent cost of control (red curves) also rises with control signal intensity. A) An increase in task incentives affects the payoff curve. Here, we consider a laboratory scenario in which monetary reward for each correct response shifts from a lower amount (dashed green curve) to a higher amount (solid). When integrated with cost information (red curve), this results in a shift in the EVC function (dashed blue curve to solid blue curve), and a resulting shift in the signal intensity that maximizes the EVC (dashed to solid black arrow). B) An increase in task difficulty reduces the expected payoff for any given control signal intensity (shift from dashed to solid green curve). In the present scenario this is due to a reduction in the probability, for any given signal intensity, of a correct response. The shift in the payoff curve, when integrated with cost (red curve), again yields a change in the EVC function (dashed to solid blue) and a shift in the EVC-maximizing control signal intensity (dashed to solid black arrow). C-D) Kouneiher and colleagues (2009) found that dACC activity tracked both of these EVC-relevant variables. They had participants perform a letter discrimination task and showed that dACC activity increased with the overall incentive level for the current block (C), whether or not the higher incentive was available for the current trial (Standard trials did not offer additional incentives for correct performance but Bonus trials did). They also found that dACC was modulated by the difficulty of a given trial (D; Default trials always had the same response mapping, obviating any additional letter discrimination, Task trials required using a letter discrimination rule based on the letter color, and this color-rule mapping was either stable throughout the session [Baseline/Contextual] or varied by block [Episodic]). See Sections 6 and 7 for additional details. Panels C-D reprinted by permission from Macmillan Publishers Ltd: Nature Neuroscience, Kouneiher et al. (2009), © (2009).