Learning prediction error neurons in a canonical interneuron circuit

Abstract

Sensory systems constantly compare external sensory information with internally generated predictions. While neural hallmarks of prediction errors have been found throughout the brain, the circuit-level mechanisms that underlie their computation are still largely unknown. Here, we show that a well-orchestrated interplay of three interneuron types shapes the development and refinement of negative prediction-error neurons in a computational model of mouse primary visual cortex. By balancing excitation and inhibition in multiple pathways, experience-dependent inhibitory plasticity can generate different variants of prediction-error circuits, which can be distinguished by simulated optogenetic experiments. The experience-dependence of the model circuit is consistent with that of negative prediction-error circuits in layer 2/3 of mouse primary visual cortex. Our model makes a range of testable predictions that may shed light on the circuitry underlying the neural computation of prediction errors.

Keywords: neural circuits; neuroscience; none; prediction-error neurons; predictive processing; sensorimotor processing; synaptic plasticity; visual system.

Figure 1.. Balancing excitation and inhibition gives rise to negative prediction-error neurons.

(a) Network model with excitatory PCs and inhibitory PV, SOM and VIP neurons. Connections from PCs not shown for the sake of clarity. Somatic compartment of PCs, SOM and PV neurons receive visual input, apical dendrites of PCs and VIP neurons receive a motor-related prediction thereof. Connections marked with an asterisk undergo experience-dependent plasticity. (b) During plasticity, the network is exposed to a sequence of feedback (coupled sensorimotor experience) and playback phases (black square, visual input not predicted by motor commands). Stimuli last for 1 s and are alternated with baseline phases (absence of visual input and motor predictions). (c) Left: Before plasticity, somatic excitation (light red) and inhibition (light blue) in PCs are not balanced. Excitatory and inhibitory currents shifted by ±20 pA for visualization. The varying net excitatory current (gray) causes the PC population rate to deviate from baseline. Right: Response relative to baseline ($\mathrm{\Delta }R/R$) of all PCs in feedback (FB), mismatch (MM) and playback (PB) phase, sorted by amplitude of mismatch response. None of the PCs are classified as nPE neurons (indicated by gray shading to the right). (d) Same as in (c) after plasticity. Somatic excitation and inhibition are balanced. PC population rate remains at baseline. All PCs classified as nPE neurons (also indicated by black shading to the right). (e) Left: Mismatch response increases with the difference between visual and motor input. Right: nPE neuron response during playback does not change with the difference between visual and motor input but remains at baseline.

Figure 1—figure supplement 1.. Learning prediction-error circuits with different forms of homeostatic plasticity.

(a) Network model as in Figure 1. Connections colored and marked with an asterisk undergo experience-dependent plasticity. (b) PCs receive visual input. Connections onto PCs follow an inhibitory plasticity rule akin to Vogels et al., 2011 (triangle). SOM→PV and VIP→PV synapses approximate a back-propagation of error (diamond). The averaged weights converge to a steady-state. Weights are normalized to the theoretically derived values for nPE neurons (see Materials and methods). (c) Same as in (b) but SOM→PV and VIP→PV synapses change in proportion to the difference between the excitatory recurrent drive onto PV neurons and a target value (square). (d) Same as in (b) but visual drive onto PCs is absent. SOM→PV and VIP→PV synapses follow an inhibitory plasticity rule akin to Vogels et al., 2011 (triangle). Connections from PCs onto PV neurons establish a baseline for PV neurons by an anti-Hebbian plasticity rule (inverted triangle) .

Figure 1—figure supplement 2.. VIP→PV synapses are not required for the formation of nPE neurons.

(a) Network model as in Figure 1 but without VIP→PV synapses. PV neurons receive visual input. (b) Population response ($\mathrm{\Delta }R/R$) of PCs in feedback (dark gray) and playback phase (light gray) for varying SOM→PV (top), SOM→VIP (middle) and VIP→SOM (bottom) connections. For all values tested, firing rate during feedback and playback deviates from baseline. (c) Response ($\mathrm{\Delta }R/R$) of all PCs in feedback, mismatch and playback phase, sorted by amplitude of mismatch response. Most PCs change their firing rate only mildly in feedback and/or playback phase. As indicated by the gray/black shading to the right, many of the PCs are classified as nPE neurons. (d–f) Same as in (a) but PV neurons receive motor predictions. (f) All PCs change their firing rate in response to all stimulation patterns. None of the PCs are classified as nPE neurons (indicated by gray shading to the right). .

Figure 1—figure supplement 3.. Balancing excitation, somatic and dendritic inhibition gives rise to nPE neurons in a model in which an excess of dendritic inhibition is forwarded to the soma.

Network model, its inputs and the training set are shown in Figure 1. Model setup modified to enable the formation of nPE neurons while abiding to Dale’s law: PCs receive 0.5 x visual input. External excitatory input onto the dendrites is set to 2.8 $s^{-1}$ to mitigate an excess of dendritic inhibition during baseline. Additional non-linearity for synapses from SOM neurons onto the apical dendrites of PCs: $\mathrm{\Delta }{w}_{\mathrm{DS}}\propto \sigma (A_{\mathrm{D}})\cdot A_{\mathrm{D}}\cdot r_{\mathrm{S}}$, where $A_{\mathrm{D}}$ denotes the total dendritic activity and $\sigma$ is a sigmoid function given by $\sigma (A_{\mathrm{D}})=0.5\cdot \left(\tanh (A_{\mathrm{D}}/3.5)+1\right)$. (a) Before plasticity, somatic excitation (light red) and inhibition (light blue) in PCs are not balanced. Excitatory and inhibitory currents are shifted by ±20 pA for visualization. The varying net excitatory current (gray) causes the PC population rate to deviate from baseline. (b) Left: Response ($\mathrm{\Delta }R/R$) of all PCs in feedback, mismatch and playback phase, sorted by amplitude of mismatch response. All PCs change their firing rate in response to all stimulation patterns. None of the PCs are classified as nPE neurons (indicated by gray shading to the right). Right: Population responses of PV, SOM and VIP neurons in all phases. Responses are normalized between −1 and 1 such that baseline is zero. (c) Same as in (a) after plasticity. Somatic excitation and inhibition are balanced. PC population rate remains at baseline. (d) Same as in (b) after plasticity. Almost all PCs classified as nPE neurons (indicated by black/gray shading to the right). PV neurons are less active during the playback phase than during the feedback phase.

Figure 2.. Multi-pathway balance of excitation and inhibition in different nPE neuron circuits.

(a) Excitatory, inhibitory, disinhibitory and dis-disinhibitory pathways onto PCs that need to be balanced in nPE neuron circuits. Input to the soma of PCs and PV neurons is varied (c–f). SOM neurons receive visual input, VIP neurons receive a motor-related prediction. (b) Test stimuli: Feedback (FB), mismatch (MM) and playback (PB) phases of 1 s each. (c) PCs and PV neurons receive visual input (left, top). When all visual (V) and motor (M) pathways are balanced (left, bottom), PCs act as nPE neurons (right). PV neuron activity increases in both feedback and playback phases. Responses normalized between −1 and 1 such that baseline is zero. (d) Same as in (c) but PV neurons receive motor predictions. (e) Same as in (c) but PCs receive no visual input. PV neurons remain at baseline in the absence of visual input to the soma of PCs. (f) Same as in (c) but PCs receive no visual input and PV neurons receive motor predictions. PV neurons remain at baseline in the absence of visual input to the soma of PCs.

Figure 2—figure supplement 1.. Multi-pathway balance of excitation and inhibition in different nPE neuron circuits with both visual and motor input onto PV neurons.

(a) Excitatory, inhibitory, disinhibitory and dis-disinhibitory pathways onto PCs that need to be balanced in nPE neuron circuits. Input to the soma of PCs and PV neurons is varied (c–d). SOM neurons receive visual input, VIP neurons receive a motor-related prediction thereof. (b) Test stimuli: Feedback (FB), mismatch (MM) and playback (PB) phases of 1 s each. (c) PCs receive visual input (left, top). When all visual (V) and motor (M) pathways are balanced (left, bottom), PCs act as nPE neurons (right). PV neuron activity increases in both feedback and playback phases. Responses normalized between −1 and 1 such that baseline is zero. (d) Same as in (c) but PCs receive no visual input. PV neurons remain at baseline in the absence of visual input to the soma of PCs.

Figure 3.. Simulated optogenetic manipulations of PV, SOM and VIP neurons disambiguate prediction-error circuits.

(a) Left: nPE neuron circuit in which PCs and PV neurons receive visual input. Inactivation (middle) or activation (right) of PV (first row), SOM (second row) or VIP neurons (third row). Optogenetic manipulations change responses of nPE neurons (Ctrl) in feedback, mismatch and playback phases. Responses normalized between −1 and 1 such that baseline is zero. Inactivation input is -$8s^{-1}$. Activation input is $5s^{-1}$. (b) Same as in (a) but PV neurons receive motor-related prediction. (c) Same as in (a) but PCs receive no visual input. (d) Same as in (a) but PCs receive no visual input and PV neurons receive a motor-related prediction.

Figure 3—figure supplement 1.. Net currents in PCs after in/activation of PV, SOM or VIP neurons elucidate prediction-error circuits.

(a) nPE neuron circuit in which PCs and PV neurons receive visual input. In/activation of PV, SOM or VIP neurons changes net current in nPE neurons in feedback (blue), mismatch (red), playback (orange) and baseline (green) phase. Gray shading denotes currents below rheobase of PC. (b) Same as in (a) but PV neurons receive motor-related prediction. (c) Same as in (a) but PCs receive no visual input. (d) Same as in (a) but PCs receive no visual input and PV neurons receive a motor-related prediction.

Figure 4.. Fraction of nPE neurons depends on SOM and VIP neuron inputs.

(a) Network model with excitatory PCs and inhibitory PV, SOM and VIP neurons. Connections from PCs not shown for the sake of clarity. Somatic compartment of PCs, PV neurons, a fraction f of SOM neurons and a fraction $(1-f)$ of VIP neurons receive visual input. The remaining SOM and VIP neurons receive motor predictions. (b) Response relative to baseline ($\mathrm{\Delta }R/R$) of all PCs in feedback, mismatch and playback phases, sorted by amplitude of mismatch response. The fraction of nPE neurons that develop during learning decreases with f (also indicated by black and gray shading to the right). The increasing fraction of non-nPE neurons comprises neurons that remain at their baseline in all three phases, show a suppression during mismatch or develop into positive prediction-error neurons that respond only during playback.

Figure 5.. Experience-dependence of nPE and PV neurons.

(a) The network is either exposed to a sequence of baseline, feedback and playback phases (quasi-natural training, QT), to baseline phases and phases during which the visual inputs and motor-related predictions are statistically independent (random gain training, RT) or perfectly coupled sensorimotor experience (coupled training, CT) (b) The number of nPE neurons that develop during learning (top) and their mismatch responses (bottom) are smaller for RT than for QT networks. 90% of SOM and 50% of VIP neurons receive visual input. (c) Population response ($\mathrm{\Delta }R/R$) of PCs, PV, SOM and VIP neurons during mismatch phase. SOM and VIP neurons show the same mismatch response for QT and RT, PCs and PV neurons show stronger responses in QT than in RT. 90% of SOM and 50% of VIP neurons receive visual input. (d) Responses during mismatch (top) and playback (bottom) for QT and CT networks. CT networks can exhibit a decrease in activity during playback phase. Connections from VIP to PV neurons are non-plastic and fixed to −0.3.

Figure 5—figure supplement 1.. Coupled-trained networks can produce nPE neurons that decrease their activity in playback phase.

(a) During plasticity, the network is exposed to a sequence of feedback phases only, representing perfectly coupled sensorimotor experience. Network model shown in Figure 1. Connections from VIP to PV neurons are non-plastic. (b–c) Model in which an excess of dendritic inhibition does not affect the soma of PCs. Connection strength from VIP to PV neurons fixed to −0.3. (b) Response ($\mathrm{\Delta }R/R$) of all PCs in feedback, mismatch and playback phase, sorted by amplitude of mismatch response. All PCs increase their activity during mismatch phase but decrease their firing rate during playback phase. The decrease of PC activity during playback is a result of an excess of somatic inhibition mediated by PV neurons. (c) Population responses of PV, SOM and VIP neurons in all phases. Responses normalized between −1 and 1 such that baseline is zero. (d–e) Model in which an excess of dendritic inhibition is forwarded to the soma of PCs. Connection strength from VIP to PV neurons fixed to −0.18. External excitatory input onto the dendrites is set to 2.8 $s^{-1}$ to mitigate an excess of dendritic inhibition during baseline. (d) Same as in (b). The decrease of PC activity during playback is a result of an excess of dendritic inhibition mediated by SOM neurons. (e) Same as in (c). PV neurons are less active during the playback phase than during the feedback phase.

Figure 6.. Learning nPE neurons by biologically plausible learning rules.

(a) Left: Network model as in Figure 1. Connections marked with symbols undergo experience-dependent plasticity. Connections onto PCs follow an inhibitory plasticity rule akin to Vogels et al., 2011 (triangle). SOM→PV and VIP→PV synapses change in proportion to the difference between the excitatory recurrent drive onto PV neurons and a target value (square). Right: During plasticity, the network is exposed to a sequence of feedback (coupled sensorimotor experience) and playback phases (black square, visual input not predicted by motor commands). Stimuli last for 1 s and are alternated with baseline phases (absence of visual input and motor predictions). (b) Left: Before plasticity, somatic excitation (light red) and inhibition (light blue) in PCs are not balanced. Excitatory and inhibitory currents shifted by ±20 pA for visualization. The varying net excitatory current (gray) causes the PC population rate to deviate from baseline. Right: Response relative to baseline ($\mathrm{\Delta }R/R$) of all PCs in feedback, mismatch and playback phases, sorted by amplitude of mismatch response. None of the PCs are classified as nPE neurons (indicated by gray shading to the right). (c) Same as in (b) after plasticity. Somatic excitation and inhibition are balanced. PC population rate remains at baseline. All PCs classified as nPE neurons (also indicated by black shading to the right).

Figure 6—figure supplement 1.. Learning nPE neurons by biologically plausible learning rules in networks without visual input at the soma of PCs.

(a) Network model as in Figure 1. Connections marked with symbols undergo experience-dependent plasticity. Inhibitory connections onto PCs and PV neurons follow an inhibitory plasticity rule akin to Vogels et al., 2011 (triangle). Synapses from PCs onto PV neurons follow an anti-Hebbian plasticity rule (inverted triangle). (b) Left: Before plasticity, somatic excitation (light red) and inhibition (light blue) at PCs are not balanced. Excitatory and inhibitory currents are shifted by ±20 pA for visualization. The varying net excitatory current (gray) causes the PC population rate to deviate from baseline. Right: Response relative to baseline ($\mathrm{\Delta }R/R$) of all PCs in feedback, mismatch and playback phase, sorted by amplitude of mismatch response. None of the PCs are classified as nPE neurons (indicated by gray shading to the right). (c) Same as in (b) after plasticity. Somatic excitation and inhibition are balanced. PC population rate remains at baseline. All PCs classified as nPE neurons (also indicated by black shading to the right).

Appendix 2—figure 1.. Multi-pathway balance of excitation and inhibition in different pPE neuron circuits.

(a) Excitatory, inhibitory, disinhibitory and dis-disinhibitory pathways onto PCs that need to be balanced in pPE neuron circuits. Input to the soma of PCs and PV neurons is varied (c–f). VIP neurons receive visual input, SOM neurons receive a motor-related prediction thereof. (b) Test stimuli: Feedback (FB), mismatch (MM) and playback (PB) phases of 1 s each. (c) PCs receive visual input. PV neurons receive visual and motor inputs (left, top). When all visual (V) and motor (M) pathways are balanced (left, bottom), PCs act as pPE neurons (right). PV neuron activity increases in both feedback and playback phases but remains at baseline during mismatch. Responses normalized between −1 and 1 such that baseline is zero. (d) Same as in (c) but PV neurons receive motor predictions only. (e) Same as in (c) but PCs receive no visual input. PV neurons remain at baseline in the absence of visual input to the soma of PCs during feedback and mismatch. (f) Same as in (c) but PCs receive no visual input and PV neurons receive motor predictions only. PV neurons remain at baseline in the absence of visual input to the soma of PCs during feedback and mismatch.

Appendix 2—figure 2.. Balancing excitation and inhibition gives rise to positive prediction-error neurons.

(a) Network model with excitatory PCs and inhibitory PV, SOM and VIP neurons. Connections from PCs not shown for the sake of clarity. Somatic compartment of PCs and VIP receive visual input, apical dendrites of PCs, SOM and PV neurons receive a motor-related prediction thereof. Connections marked with an asterisk undergo experience-dependent plasticity. (b) During plasticity, the network is exposed to a sequence of feedback (coupled sensorimotor experience) and mismatch phases (black square, no visual flow despite motor-related predictions). Stimuli last for 1 s and are alternated with baseline phases (absence of visual input and motor predictions). (c) Left: Before plasticity, somatic excitation (light red) and inhibition (light blue) in PCs are not balanced. Excitatory and inhibitory currents shifted by ±20 pA for visualization. The varying net excitatory current (gray) causes the PC population rate to deviate from baseline. Right: Response relative to baseline ($\mathrm{\Delta }R/R$) of all PCs in feedback (FB), mismatch (MM) and playback (PB) phase, sorted by amplitude of mismatch response. None of the PCs are classified as pPE neurons. (d) Same as in (c) after plasticity. Somatic excitation and inhibition are balanced. PC population rate fluctuates around baseline. All PCs classified as pPE neurons. Connection strength from VIP onto SOM neurons is set to 0.8.