Neural variability in the medial prefrontal cortex contributes to efficient adaptive behavior

Abstract

Neural variability, i.e. random fluctuations in neural activity, is a ubiquitous and sizable brain feature that impacts behavior. Its functional role however remains unclear and neural variability is commonly viewed as a nuisance factor degrading behavioral efficiency. Using functional magnetic resonance imaging in humans and computational modeling, we show here that neural variability provides a solution to the open issue regarding how the brain produces efficient adaptive behavior in uncertain and changing environments without facing computational complexity problems. We found that neural variability in the medial prefrontal cortex (mPFC) enables decision-making processes in the mPFC to produce near-optimal behavior in uncertain and ever-changing environments without involving complex computations known in such environments to rapidly become computationally intractable. The results thus suggest that in the same way as genetic variability contributes to adaptive evolution, neural variability contributes to efficient adaptive behavior in real-life environments.

Fig. 1. Inference models and experimental paradigm of adaptive behavior.

A, left model comprising three hierarchically organized probabilistic inference levels. First-level: inferences about environment latent states z_t characterizing current external contingencies (stimulus-action-outcome contingencies: s_t-a_t-r_t). Second-level: inferences about change probabilities τ_t (i.e., volatility) in current latent states. Third-level: inferences about the change rate ν of volatility. Rate ν is assumed to be constant across time but its estimate may vary across time. Middle, model comprising only the two lower inference levels: volatility τ is assumed to be constant but its estimates may vary across time. Right, models comprising only the lowest inference level: the environment latent state z is assumed to remain unchanged over time but beliefs about its identity may vary over time. The Weber-variability model comprises only the first inference level but these first-order inferences undergo computational imprecisions in agreement with Weber’s Law, providing the necessary flexibility to adapt to changing environments. B trial structure of the two-arm bandit task. In every trial, participants chose one of the two visually presented arms (square vs. circle with randomized left-right positions) by pressing the corresponding hand-held response button. 0.4–3.8 s later (jittered) and contingent upon participants’ choices, they received a visually presented reward (1 euro shown) or not (red cross) (duration: 700 ms). Inter-trial intervals were jittered from 0.5 to 3.9 seconds. C Reward probabilities associated with arms (15% vs. 85%) reversed with probability 0.05 (low volatility episodes), 0.07 (middle volatility episodes), and 0.1 (high volatility episodes). Episode order was pseudo-randomized.

Fig. 2. Model fitting and simulation of participants’ adaptive performances.

A Bayesian model comparison given participants’ choices across the third-/second-order volatility inference model, Weber-variability model, and best-fitting RL model (noisy RW-RL, Supplementary Fig. 2). Bars show exact model posterior probabilities over the n = 22 participants (“Methods” section). Model exceedance probabilities are shown in brackets. Error bars: Bayesian estimates of model posterior probability standard deviations. The Weber-variability model fitted decisively better than the other models. B Confusion matrices from the model recovery procedure across the same models. Large matrix: exact model posterior probability given models’ simulated performances; small matrix: model exceedance probability. Each model fitted its own simulated performance in the task decisively better than the other models. C simulations of fitted model performances compared to participants’ performances around reversals in high volatility episodes. Shaded areas: s.e.m. across participants. Statistically significant differences (two-sided T-tests over three consecutive trials, d.f. = 21) are shown. Left: *p = 0.029, ****p = 0.000051; middle left: **p = 0.0098, ****p = 0.000952; middle right: all ps > 0.09. Right: all ps < 0.000015. Only the Weber-variability model reproduced participants’ choices with no significant deviations. Note that the best RL model (noisy RW-RL) dramatically failed (see Supplementary Note 1 for explanation). Source data are provided as a Source Data file.

Fig. 3. Empirical characteristics of Weber variability.

A Bayesian model comparison given participants’ choices (computed over the n = 22 participants) when models comprise no choice history accounts (left, same data as in Fig. 2A), comprise repetition biases (middle) and choice trace biases (right). Bars show exact model posterior probabilities. Model exceedance probabilities are shown in brackets. Error bars: Bayesian estimates of model posterior probability standard deviations. In every case, the Weber-variability model fitted decisively better than the other models, indicating that fitted Weber variability was unrelated to such choice history effects. B One-way ANOVA of best-fitting realizations $U_{fit}^t$ of Weber variability stochastic component $u_t$ over the n = 22 participants and factoring choice history from trial t-3 to t-1 (Switch vs. Repeat responses relative to preceding trials) into an 8-level fixed-effect factor. Bars show the means over participants. Error bars are s.d. across trials within participants. The choice history factor accounted for only η² = 17% of $U_{fit}^t$ total variance, indicating that 83% of $U_{fit}^t$ total variance was unrelated to any three-fold choice history. C autocorrelations of best-fitting realizations $U_{fit}^t$ of Weber variability stochastic components $u_t$ across successive trials averaged over participants (n = 22). These realizations $U_{fit}^t$ showed virtually no autocorrelations (all R² < 0.005). Bars and error bars are mean ± s.e.m. over the n = 22 participants. D Empirical distribution of best-fitting realizations $U_{fit}^t$ across trials and participants. Prior generative distribution $u_t$ is uninformative, i.e., uniform over [0;1]. The posterior distribution is obtained from marginalizing over parameter spaces and particle trajectories from particle filters (see “Methods” section). Note that this empirical posterior distribution is approximately Gaussian, centered on its mean 0.5, as expected from averaging over a series of independent random variables. E Mean ± s.d. (over the n = 22 participants) of empirical best-fitting realizations $U_{fit}^t$ distributions along experimental blocks (scanning runs) and fMRI sessions. Note the lack of any temporal order effects (F(5,105) = 1.737, p = 0.1544). Source data are provided as a Source Data file.

Fig. 4. Brain activations associated with choice computations from corrupted beliefs.

A unique MRI bold activation cluster in the whole-brain analysis associated with choice computations at choice time, i.e., exhibiting jointly a negative linear effect $\left(-\left({\tilde{B}}_{ch}-{\tilde{B}}_{unch}\right)\right)$ and a positive quadratic effect ${\left({\tilde{B}}_{ch}-{\tilde{B}}_{unch}\right)}^2$ of chosen-relative-to-unchosen beliefs undergoing Weber variability (dark blue, conjunction analysis, voxel-wise threshold p < 0.001, cluster-wise FWE-corrected p < 0.05); Linear and quadratic effects capture signed and unsigned differences respectively between chosen and unchosen beliefs (see text). Activations are superimposed on the MNI template sagittal and axial slices centered on the unsigned difference activation peak. B MRI activity at choice time averaged over the activation cluster shown in (A) plotted against chosen-relative-to-unchosen beliefs undergoing Weber variability and factoring out RTs and either the quadratic effect (left) or the linear effect (right). Note both the predicted negative linear effect and positive quadratic effect. C Full variance analyses at choice time over the activation cluster shown in (A) comprising the signed ${\tilde{B}}_{ch}-{\tilde{B}}_{unch}$ and unsigned ${\left({\tilde{B}}_{ch}-{\tilde{B}}_{unch}\right)}^2$ differences between chosen and unchosen beliefs undergoing Weber variability, along with Weber variability (and with or without Reaction Times) as regressors. Data points show individual subjects’ data. Note that the Weber variability regressor captured no residual variances. Left graph: ***from left to right bars: p < 0.00001, p = 0.000038, p = 0.00007, p = 0.148; right graph: p < 0.00001, p = 0.000012, p = 0.138 (one-sample two-sided T-test, d.f. = 21). D Best-fitting realizations $U_{fit}^t$ of Weber variability stochastic components $u_t$ plotted against activation residuals averaged over the cluster shown in (A) and normalized by Weber variability deterministic component ${\mu }_{fit}+{\lambda }_{fit}{d}_t$. Activation residuals were computed from the regression analysis comprising Reaction Times and both $\left(-\left(B_{ch}-B_{unch}\right)\right)$ and ${\left(B_{ch}-B_{unch}\right)}^2$ as regressors with Weber variability ${ϵ}_t^{fit}$ removed from current beliefs when forming these regressors. The graph indicates that the best-fitting realizations $U_{fit}^t$ that corrupt beliefs driving choices reflected neural fluctuations, corrupting belief updating in the pre-SMA and ACC. All error bars are s.e.m. across participants. ***p < 0.001 (one-sample two-sided T-test, d.f. = 21). Source data are provided as a Source Data file.

Fig. 5. Neural fluctuations associated with Weber variability.

A MRI bold activation cluster in the dmPFC associated with Weber variability ${ϵ}_t^{fit}=\left({\mu }_{fit}+{\lambda }_{fit}{d}_t\right).U_{fit}^t$ at choice time (light-blue, voxel-wise p < 0.001, cluster-wise FWE-corrected p < 0.05), superimposed on the MNI template sagittal slice centered on the voxel exhibiting the maximal correlation. B Weber-variability -related activations averaged over the activation cluster shown in (A) at outcome and choice time (mean ± s.e.m. over the n = 22 participants from a leave-one-out procedure removing selection biases). Note the expected marginal correlation at outcome time ($\mathit{~}$ p = 0.059; ****p < 0.0000001; two-sided one-sample T-tests). C Bayesian model comparison over the n = 22 participants between Weber variability ${ϵ}_t^{fit}$ and its sole deterministic component ${\mu }_{fit}+{\lambda }_{fit}{d}_t$ as concurrent models of these activations at both outcome and choice time. Bars are model posterior probabilities; error bars are Bayesian estimates of model posterior probability standard deviations. Model exceedance probability P_exc is shown. To remove selection biases, the Bayesian model comparison was performed only on voxels correlating with both ${ϵ}_t^{fit}$ and ${\mu }_{fit}+{\lambda }_{fit}{d}_t$ (voxel-wise threshold p < 0.001) and through a leave-one-out procedure. D, left: model with $U_{fit}^t$ as the unique regressor (**p = 0.0045; ****p = 0.0000051; one-sample two-sided T-tests, d.f. = 21); right: full variance analyses over these voxels comprising Weber variability ${ϵ}_t^{fit}$ as regressor of interest and factoring out deterministic component ${\mu }_{fit}+{\lambda }_{fit}{d}_t$ ₍shown on the plot) along with other variables of no interest, including RTs, response switches, and RL variables (**p = 0.0021, ***p = 0.00025; one-sample two-sided T-tests, d.f. = 21). Error bars are s.e.m. across participants. dmPFC activity correlated negatively at outcome time and positively at choice time with best-fitting realizations $U_{fit}^t$ of Weber variability stochastic component $u_t$. Data points show individual subjects’ data. See supplementary Fig. 5 for additional analyses regarding individual data. Source data are provided as a Source Data file.

Fig. 6. Weber variability and volatility -related dmPFC activations.

A MRI bold activation cluster correlating with volatility estimates from the third-order volatility inference model at outcome time (dark-green, voxel-wise p < 0.001, cluster-wise FWE-corrected p < 0.05) superimposed on the MNI template sagittal slice centered on the activation peak (MNI coordinates: x,y,z = 6,17,50 mm). Light-blue area: same data as in Fig. 5A. B Volatility-related activations averaged over the dark-green cluster shown in (A) at outcome and choice time (leave-one-out procedure removing selection biases). Mean and s.e.m across the n = 22 participants (**p = 0.0023; two-sided one-sample T-tests). C Full variance analyses over the dark-green activation cluster shown in (A), comprising volatility estimates and Weber variability ${ϵ}_t^{fit}$ as regressor of interest and factoring out variables of no interest, including RTs, response switches, and RL variables. Error bars are s.e.m. across participants (n = 22). **p = 0.006, ***p = 0.0004, otherwise ps > 0.47; two-sided one-sample T-tests, d.f. =21). D Bayesian model comparison over the n = 22 participants between volatility estimates and Weber variability ${ϵ}_t^{fit}$ as concurrent models of both outcome- and choice-related activations in the dark-green cluster shown in (A). Bars are model posterior probabilities. Error bars are Bayesian estimates of s.d. of the model posterior probability. Model exceedance probability P_exc is indicated. All data points show individual subjects’ data. Source data are provided as a Source Data file.