An insula-frontostriatal network mediates flexible cognitive control by adaptively predicting changing control demands

Abstract

The anterior cingulate and lateral prefrontal cortices have been implicated in implementing context-appropriate attentional control, but the learning mechanisms underlying our ability to flexibly adapt the control settings to changing environments remain poorly understood. Here we show that human adjustments to varying control demands are captured by a reinforcement learner with a flexible, volatility-driven learning rate. Using model-based functional magnetic resonance imaging, we demonstrate that volatility of control demand is estimated by the anterior insula, which in turn optimizes the prediction of forthcoming demand in the caudate nucleus. The caudate's prediction of control demand subsequently guides the implementation of proactive and reactive attentional control in dorsal anterior cingulate and dorsolateral prefrontal cortices. These data enhance our understanding of the neuro-computational mechanisms of adaptive behaviour by connecting the classic cingulate-prefrontal cognitive control network to a subcortical control-learning mechanism that infers future demands by flexibly integrating remote and recent past experiences.

Figure 1. Model structure and schematic illustration of model-based analyses (N=21).

(a) The graphical representation of the belief update model (upper panel) and the flexible control model (lower panel). The two models use identical structure and inference algorithms except that the flexible control model also uses reaction time (RT) to update the belief of latent variables. Note that only the flexible control model was used in all analyses to account for individual difference of behaviour. The flexible control model uses four variables, flexible LR/volatility (α), predicted conflict (f), congruency (o, shown in grey indicating this variable is observable) and RT for each trial. The directed edges indicate the information flow. At a given trial, horizontal and top–down edges represent the estimation of flexible LR/volatility and predicted conflict level prior to stimulus presentation. Subsequently, the observed congruency and RT are used to update belief of latent variables in anticipation of the next trial (bottom-up edges). (b) A schematic illustration of the univariate and multivariate encoding analysis using the example model variable of predicted conflict level (see Methods for details). Importantly, prior to multivariate analysis, the searchlight mean activation vector was also regressed from each voxel's activation vector to ensure the orthogonality between the two analyses.

Figure 2. Experimental task and simulation and behavioural results (N=21).

(a) Example stimuli and timing of presentation. This example depicts an incongruent trial, followed by a congruent trial. (b) Individual mean model LRs, plotted as a function of run type. Each line represents a subject. (c) The time course of group mean LR and s.e.m. in the first and last 10 trials of volatile (in blue) and stable (in red) blocks. Note that in this graph, which averages over all blocks, the difference in LR at the beginning of the blocks was driven by volatile blocks 2–4 in the volatile runs, as in these blocks the LR had already been raised by preceding volatile blocks. (d) Time courses of the underlying proportion congruency (in black) and the corresponding predicted conflict level (in red) of an example run. (e) Individual RS, plotted as a function of congruency. Each line represents one subject. Note that higher RT equals lower RS. (Con=congruent trials; Inc=incongruent trials). (f) Group mean RS and s.e.m., centred across trials for each subject, plotted as a function of unsigned prediction error of congruency (control prediction error).

Figure 3. Estimation and updating of the volatility-driven flexible LR of control demand (N=21).

(a) Searchlights in the AI and adjacent IFG track the model estimates of volatility/LR (in red, P<0.05 corrected, one sample t-test). (b) The flexible control model, highlighting in red the information processing mechanisms related to the updating of LR. (c) Visualization of searchlights encoding LR (red, P<0.05 corrected) and searchlights showing an interaction between predicted conflict level and congruency, or control prediction error (green, P<0.05 corrected, one sample t-test); overlap between these independently defined effects is shown in yellow. (d) Probabilistic distribution of t-values measuring the group-level univariate effect of prediction error of congruency (in grey) and the underlying null t-distribution (in red). The t-values were calculated from searchlights in the LR-encoding cluster shown in (a). The red vertical dotted line denotes the threshold for statistical significance (P<0.05). (e) Group mean activation levels (±s.e.m.) in the left AI/IFG ROI showing significant encoding of the volatility-driven flexible LR. The trial-by-trial ROI-mean fMRI activation (in t-values, in accordance with the univariate and multivariate analyses) were binned for each individual into 20 quantiles, which are plotted as a function of 20 quantiles of the LR (left) and estimation uncertainty (right). (f) Trial-wise estimation uncertainty of a representative example subject, plotted as a function of its corresponding LR. Each trial is represented by a translucent grey disk. Although the correlation coefficient between these two model estimates is significant (r=0.32, P<0.0001), the r² statistic indicates that only 10% of variance is shared between them (for group statistics, see Supplementary Tables 5 and 6).

Figure 4. Modulation of volatility on predicted control demand (conflict level, N=21).

(a) Searchlights in the caudate track the model's prediction of conflict level (in red, P<0.05 corrected, one sample t-test) (b) The flexible control model, highlighting in red the information processing mechanisms related to the modulation of volatility on predicted conflict level. (c) Individual modulation of volatility on caudate activity-derived LR. Each horizontal bar represents a participant.

Figure 5. Titration of cognitive control by predicted conflict level (N=21).

(a) The flexible control model, highlighting in red the information processing mechanisms related to the mediation of predicted conflict level on cognitive control. (b,c) Brain regions involved in proactive/reactive control in the dlPFC (b) and ACC (c), showing a significant correlation between predicted conflict level × congruency interaction in the RSs and encoding strength of predicted conflict level (in red)/prediction error of predicted conflict level (in green), respectively. For each cluster displayed, predicted conflict level × congruency interaction in the RS is plotted as a function of cluster mean of encoding strength across subjects.