Uncertainty-modulated prediction errors in cortical microcircuits

Abstract

Understanding the variability of the environment is essential to function in everyday life. The brain must hence take uncertainty into account when updating its internal model of the world. The basis for updating the model are prediction errors that arise from a difference between the current model and new sensory experiences. Although prediction error neurons have been identified in layer 2/3 of diverse brain areas, how uncertainty modulates these errors and hence learning is, however, unclear. Here, we use a normative approach to derive how uncertainty should modulate prediction errors and postulate that layer 2/3 neurons represent uncertainty-modulated prediction errors (UPE). We further hypothesise that the layer 2/3 circuit calculates the UPE through the subtractive and divisive inhibition by different inhibitory cell types. By implementing the calculation of UPEs in a microcircuit model, we show that different cell types can compute the means and variances of the stimulus distribution. With local activity-dependent plasticity rules, these computations can be learned context-dependently, and allow the prediction of upcoming stimuli and their distribution. Finally, the mechanism enables an organism to optimise its learning strategy via adaptive learning rates.

Keywords: cells; circuits; cortex; neuroscience; none.

Figure 1.. Distributed uncertainty-modulated prediction error computation in cortical circuits.

(A) A person who learned that buses are unreliable has a prior expectation, which can be described by a wide Gaussian distribution of expected bus arrival times. When the bus does not arrive at the scheduled time, this person is not surprised and remains calm, as everything happens according to their model of the world. On the other hand, a person who learned that buses are punctual, which can be described by a narrow distribution of arrival times, may notice that the bus is late and get nervous, as they expected the bus to be punctual. This person can learn from this experience. If they always took this particular bus, and their uncertainty estimate is accurate, the prediction error could indicate that the bus schedule changed. (B) Models of uncertainty representation in cortex. Some models suggest that uncertainty is only represented in higher level areas concerned with decision-making (left). In contrast, we propose that uncertainty is represented at each level of the cortical hierarchy (right, shown is the visual hierarchy as an example). (C) A mouse learns the association between a sound (${\displaystyle a}$) and a whisker deflection (${\displaystyle s}$). The posterior parietal cortex (PPC) receives inputs from both somatosensory and auditory cortex. (D) The whisker stimulus intensities are drawn from a Gaussian distribution with mean μ and standard deviation ${\displaystyle \sigma }$. (E) Negative (left) and positive (right) prediction error circuit consisting of three cell types: layer 2/3 pyramidal cells (triangle), somatostatin-positive interneurons (SST, circle) and parvalbumin-positive interneurons (PV). SSTs represent the mean prediction in the postive circuit and the stimulus in the negative circuit, and PVs represent the variance.

Figure 2.. SSTs learn to represent the mean context-dependently.

Illustration of the changes in the positive prediction error circuit. Thicker lines denote stronger weights. (B) Two different tones (red, orange) are associated with two somatosensory stimulus distributions with different means (red: high, orange: low). (C) SST firing rates (mean and std) during stimulus input. (D) SST firing rates over time for low (orange) and high (red) stimulus means. (E) Weights (mean and std) from sound a to SST for different values of μ. (F) SST firing rates (mean and std) for different values of μ. Mean and std were computed over 1000 data points from timestep 9000–10,000.

Figure 3.. PVs learn to estimate the variance context-dependently.

(A) Illustration of the changes in the positive prediction error circuit. Thicker lines denote stronger weights. (B) Two different tones (purple, green) are associated with two somatosensory stimulus distributions with different variances (purple: high, green: low). (C) Weights from sound a to PV over time for two different values of stimulus variance (high: ${\displaystyle \sigma =0.8}$ [purple], low: ${\displaystyle \sigma =0.4}$ [green]). (D) PV firing rates over time given sound input (without whisker stimulus input) for low (green) and high (purple) stimulus variance. (E) PV firing rates (mean and std) given sound input and whisker stimuli for low and high stimulus variance. (F) PV firing rates (mean and std) during sound and stimulus input. (G) Weights (mean and std) from sound ${\displaystyle a}$ to PV for different values of ${\displaystyle \sigma }$. (H) PV firing rates (mean and std) given sound input for different values of ${\displaystyle {\sigma }^2}$. Mean and std were computed from 150,000 data points from timestep 450,000–600,000.

Figure 3—figure supplement 1.. Different choice of supralinear activation function for PV.

Learning the variance in the positive prediction error circuit with PVs with a power activation function (exponent = 3.0). (A, B) are analogous to Figure 3G and H, and the circuit is the same except that the activation function of the PVs (${\displaystyle {\varphi }_{PV}(x)}$) has an exponent of 3.0 instead of 2.0. (C, D) are zoomed-out versions of (A, B).

Figure 3—figure supplement 2.. Plastic weights from SST to PV learn to match weights from s to PV.

With inhibitory plasticity, weights from SST to PV can be learned. This figure shows that the weight from SST to PV (${\displaystyle w_{\mathrm{P}\mathrm{V},\mathrm{S}\mathrm{S}\mathrm{T}}}$) is equal to the weight from ${\displaystyle s}$ to PV (${\displaystyle w_{\text{PV},s}}$). The inhibitory plasticity rule is described in the Appendix.

Figure 3—figure supplement 3.. PVs learn to represent the variance given an associative cue in the negative prediction error circuit.

(A) Illustration of the changes in the negative prediction error circuit. Thicker lines denote stronger weights. (B) Two different tones (purple, green) are associated with two somatosensory stimulus distributions with different variances (purple: high, green: low). (C) Weights from sound ${\displaystyle a}$ to PV over time for two different values of stimulus variance (high: ${\displaystyle \sigma =0.8}$ [purple], low: ${\displaystyle \sigma =0.4}$ [green]). (D) PV firing rates over time given sound input (without stimulus input) for low (green) and high (purple) stimulus variance. (E) PV firing rates (mean and std) given sound input for low and high stimulus variance. (F) PV firing rates (mean and std) during sound and stimulus input. (G) Weights from sound ${\displaystyle a}$ to PV for different values of ${\displaystyle \sigma }$ (mean and std). (H) PV firing rates given sound input for different values of ${\displaystyle {\sigma }^2}$ (mean and std).

Figure 4.. Calculation of the UPE in layer 2/3 error neurons.

(A) Illustration of the negative prediction error circuit. (B) Distributions with different standard deviations ${\displaystyle \sigma }$. (C) Illustration of the positive prediction error circuit. (D) Firing rate of the error neuron in the negative prediction error circuit (UPE) as a function of ${\displaystyle \sigma }$ for two values of ${\displaystyle |s-\mu |}$ after learning μ and ${\displaystyle \sigma }$. (E) Rates of both UPE⁺ and UPE⁻-representing error neurons with a nonlinear activation function, where ${\displaystyle k=2.0}$, as a function of the difference between the stimulus and the mean (${\displaystyle s-\mu }$). (F) Firing rate of the error neuron in the positive prediction error circuit (UPE⁺) as a function of ${\displaystyle \sigma }$ for two values of ${\displaystyle |s-\mu |}$ after learning μ and ${\displaystyle \sigma }$. (G-I) same as (D-F) for error neurons with ${\displaystyle k=2.5}$ with the same legend as in (D-F).

Figure 4—figure supplement 1.. Learning the weights from the SSTs to the UPE neurons.

This figure shows that the weights from the SSTs to the UPEs in both the positive (left) and the negative (right) prediction error circuit can be learned with inhibitory plasticity to match the weights from the stimulus representation ${\displaystyle s}$ to the UPEs. The inhibitory plasticity rule is described in the Appendix.

Figure 5.. Learning the mean representation with UPEs.

(A) Illustration of the circuit. A representation neuron (turquoise) receives input from both positive and negative prediction error circuits (UPE⁺ and UPE⁻) and projects back to them. The UPE⁻ has a negative impact on the firing rate of the representation neuron (${\displaystyle r_R}$). A weight ${\displaystyle w_{R,a}}$ from the higher level representation of the context given by sound ${\displaystyle a}$ is learned. (B) Weights ${\displaystyle w_{R,a}}$ over time for different values of μ (${\displaystyle \mu \in [1,3,5]}$). (C) R firing rates given sound input for different values of μ (mean and std over 50,000 data points from timestep 50,000–100,000,, the end of the simulation). (D) Activity of the different cell types (PV: light green, R: turquiose, UPE: black) and whisker stimulus samples (grey dots) over time. Learning the mean representation with PVs (light green) reflecting the MSE at the beginning, which is compensated by nonlinear activation of L2/3 neurons (black). The evolution of the mean rate of neuron ${\displaystyle R}$ (turquoise) is similar to the perfect case in (E). (E) Same colour code as in (D). Inset shows comparison to (D) Learning the mean representation assuming PVs (light green) perfectly represent the variance.

Figure 5—figure supplement 1.. PV firing rates are proportional to the variance in the recurrent circuit model.

Weights from ${\displaystyle a}$ to PV as a function of ${\displaystyle \sigma }$ in the positive (A) and negative (C) prediction error subcircuit. PV firing rates as a function of ${\displaystyle {\sigma }^2}$ in the positive (B) and negative (D) prediction error circuit.

Figure 6.. Cell-type-specific experimentally testable predictions.

(A) Illustration of the two experienced stimulus distributions with different variances that are associated with two different sounds (green, purple). The presented mismatch (MM) stimulus (black) is larger than expected (positive prediction error). (B-F) Simulated firing rates of different cell types to positive prediction errors when a sound associated with high (purple) or low (green) uncertainty is presented. (G) As in (A) the presented mismatch stimulus (grey) is smaller than expected (negative prediction error). (H-L) Firing rates of different cell types to the negative mismatch when a sound associated with high (purple) or low (green) uncertainty is presented. Because the firing rate predictions are equal for PV⁺ and PV⁻, we only show the results for PV⁺ in the figure.

Figure 7.. Effective learning rate is automatically adjusted with UPEs.

(A, B) Firing rate over time of the representation neuron in a circuit with uncertainty-modulated prediction errors (gold) and in a circuit with unmodulated errors (black) in a low uncertainty setting (A) and a high uncertainty setting (B). (C) Standard deviation of the firing rate of the representation neuron in the low uncertainty setting (inset has a different scale, outer axis scale matches the one in D). (D) Standard deviation of the firing rate of the representation neuron in the high uncertainty setting. (E) Standard deviation of the firing rate ${\displaystyle r_R}$ as a function of the standard deviation of the presented stimulus distribution ${\displaystyle {\sigma }_s}$. Standard deviations were computed over 100,000 data points from timestep 100,000–200,000.

Figure 8.. UPEs ensure uncertainty-based weighting of prior and sensory information.

(A) Illustration of the hierarchical two-area model. A representation neuron (${\displaystyle r}$) in area ${\displaystyle \ell }$ receives positive and negative UPEs from the area ${\displaystyle \ell -1}$ below (sensory prediction error, as in Figure 5), and positive and negative UPEs from the same area (representation prediction error) with different signs, see Equation 5. In this example, the uncertainty in area ${\displaystyle \ell -1}$ corresponds to the sensory uncertainty ${\displaystyle {\sigma }_s^2}$, and the uncertainty in the area ${\displaystyle \ell }$ above, corresponds to the prior uncertainty ${\displaystyle {\sigma }_p^2}$. Both uncertainties are represented by PV activity in the respective area. (B, C) The simulation results from the hierarchical circuit model yield an uncertainty-based weighting of prior and sensory information as in Equation 4. (B) The activity ${\displaystyle r}$ of the representation neuron as a function of the stimulus ${\displaystyle s}$ for different amounts of prior uncertainty ${\displaystyle {\sigma }_p^2}$, when the sensory uncertainty is fixed ${\displaystyle {\sigma }_s^2=0.5}$. The histograms show the distribution of the sensory stimulus samples (grey) and the distribution of the activity of the representation neuron (green). (C) The activity of the representation neuron as a function of the stimulus for different amounts of sensory uncertainty ${\displaystyle {\sigma }_s^2}$, when the prior uncertainty is fixed ${\displaystyle {\sigma }_p^2=0.5}$.

Figure 9.. For small β, and ws=2−ββ, the weight from a to PV approaches σ and the PV firing rate approaches σ2.