Developmental self-construction and -configuration of functional neocortical neuronal networks

Abstract

The prenatal development of neural circuits must provide sufficient configuration to support at least a set of core postnatal behaviors. Although knowledge of various genetic and cellular aspects of development is accumulating rapidly, there is less systematic understanding of how these various processes play together in order to construct such functional networks. Here we make some steps toward such understanding by demonstrating through detailed simulations how a competitive co-operative ('winner-take-all', WTA) network architecture can arise by development from a single precursor cell. This precursor is granted a simplified gene regulatory network that directs cell mitosis, differentiation, migration, neurite outgrowth and synaptogenesis. Once initial axonal connection patterns are established, their synaptic weights undergo homeostatic unsupervised learning that is shaped by wave-like input patterns. We demonstrate how this autonomous genetically directed developmental sequence can give rise to self-calibrated WTA networks, and compare our simulation results with biological data.

Figure 1. Winner-take-all architecture.

(A) Architecture of an idealised winner-take-all-network. Several excitatory neurons (red) excite a single shared inhibitory neuron, or a shared population of inhibitory neurons (blue). Each excitatory neuron receives inhibitory feedback in proportion to the average activity of the excitatory population. (B) The WTA architecture is embedded in the field of recurrent connections between a population of excitatory and inhibitory neurons. (C) Once the WTA architecture has formed, coarsely structured synaptic input drives synaptic refinement of the recurrent connections within the network.

Figure 2. Gene Regulatory Network.

(A) Schematic representation of the GRN, composed of five interacting genes that give rise to excitatory and inhibitory neurons. The identity of a neuron is determined by the genes GE and GI for excitatory or inhibitory neurons, respectively. Arrows indicate a positive effect on gene expression. (B) Lineage tree. Nodes indicate cells; boxes indicate gene expression patterns. G0 triggers the expression of G1, which characterizes the undifferentiated state of progenitor cells. After a series of symmetric divisions, G1 reaches a concentration threshold. According to fixed probabilities, G1 can then activate the differentiation toward excitatory (red) or inhibitory (blue) neurons. Alternatively, a small proportion of cells probabilistically undergoes a second round of cell division and activates gene G2, which again promotes the differentiation toward excitatory or inhibitory neurons by the expression of GE or GI. The probabilistic activation of inhibitory or excitatory genes is a simplification, but guarantees the production of a homogeneously mixed population of neurons.

Figure 3. Developmental process for building a competitive network.

A single precursor cell (A) contains the genetic code specifying the entire developmental process. (B) The precursor cell first undergoes repeated division to increase the pool of neuronal precursors (black). (C) Precursor neurons then differentiate into excitatory and inhibitory cell classes. (D) Neurite outgrowth begins to provide a scaffold for synaptic connections. (E) A network of differentiated neurons (grey) after neurite outgrowth has finished. For better visualization, examples of excitatory and inhibitory neurons are colored in red and blue, respectively. (F) Synapses (black rectangles) can form at appositions between axons and dendrites.

Figure 4. Connectivity after simulated neurite outgrowth.

(A) Comparison of connectivity statistics from Cx3D simulations (blue) with experimental data (red) from . Indicated on the vertical axis are the numbers (normalized with respect to the first bar) of synapses onto a single neuron. The individual bars show the values for the different pre- and postsynaptic neuron pairs (excitatory or inhibitory synapses onto an excitatory or inhibitory postsynaptic neuron). The numbers match in proportion, while the absolute quantities are higher in the biological data (approximately 155 vs. 3500 excitatory synapses onto a single excitatory neuron in the simulated and biological connectivity, respectively). This particular simulation consists of 250 neurons (200 excitatory and 50 inhibitory), which are randomly arranged in 3D space. (B) Histogram of the percentage of excitatory input synapses across the simulated network from (A). Each bar indicates the number of neurons that have a particular percentage of excitatory input synapses (after neurite growth and synapse formation have ended). The final distribution has a mean of 84%, which is in line with experimental assessments , –.

Figure 5. Homeostatic adaptation of neuronal firing rates during establishment of synaptic connectivity.

(A) Synaptic scaling during neurite outgrowth leads to robust average activities of both excitatory (red) and inhibitory (blue) neurons. The network consists of 250 neurons that are randomly arranged in 3D space. The horizontal axis indicates the estimated real-time when taking into account that the time constant of synaptic scaling is in the order of several hours . At (dashed line), the neurite outgrowth begins. Average firing rates of layer II/III pyramidal neurons have been shown to be smaller than 1 Hz in-vivo , . Experimental data indicates that inhibitory neurons have higher activities (Eq. 2) than excitatory neurons , , , . In this simulation there are not yet any input projections, so the activity originates solely from internally generated and random activity. (B) Total (excitatory and inhibitory) number of synapses in the network during development. New synapses are formed also after the neurons reach the target average activities, without disrupting the homeostatic adaptation process or bringing the network out of balance. These simulation results demonstrate the robustness of the synaptic scaling process during network growth.

Figure 6. Winner-take-all functionality.

(A) Weight matrix of 117 excitatory neurons in a WTA network. After learning the network exhibits a 1-dimensional neighborhood topology, as shown by the predominantly strong weights around the diagonal. This topology mirrors the neighborhood relationship of the input stimuli, which are continuously and periodically moving hills of activity. Only the excitatory connections are shown here, because the inhibitory neurons do not integrate into the neighborhood topology (see text). (B) Demonstration of WTA functionality on the network connectivity shown in (A). Neurons are ordered here such that adjacent neurons connect most strongly. The input to the network (; top row) has a hill shape, with added noise. The network response (; middle row) is a de-noised version of the input with the bump in the same location. The neuronal gain (; bottom row) is high for neurons receiving the strongest input, and low (or zero) for neurons distant from the main input to the network. The dashed horizontal line indicates a gain of 1. (C) Activity of a winning neuron (blue, solid), during presentation of its feedforward input (blue, dashed) in the same simulation as shown in (B). Recurrent connectivity amplifies the response of the neuron for the duration of the stimulus (). In contrast, a losing neuron (green, solid) receives non-zero feedforward input (green, dashed), but is suppressed due to the WTA functionality of the network. (D) Response of the same network to a different feedforward input. The recovery of a bump shaped activity can occur anywhere in the network topology.

Figure 7. Clustering and decorrelation of representations.

(A–C) Discrete input patterns give rise to clusters in the functional connectivity of the WTA network. (A) Input stimuli used in the learning process. Filled and empty spheres indicate strongly and weakly active populations, respectively. (B,C) Visualization of the network structure before and after learning. Strongly-coupled neurons are drawn close together; excitatory synaptic connections are indicated by grey links. Excitatory neurons are coloured according to their preferred input pattern (colours in A); inhibitory neurons (square) are drawn in yellow. (B) Before learning, no clustering of synaptic connections is present. (C) After learning, neurons with the same preferred stimulus are strongly interconnected. See S2 Video. (D) Before learning, the response of the network is similar across all stimuli. Shown is the scalar product between the vectors of neuronal responses to pairs of stimuli . The noise was added in order to assess the sensitivity of the network's activity to a perturbation of the input signal (see text). The high values and uniformity of scalar products in (D) indicates that network responses poorly distinguish between stimuli. (E) After learning, responses to noisy stimulus presentations are highly similar (high values of scalar product; black diagonal), whereas responses to different stimuli are decorrelated (low values of scalar product; light shading).

Figure 8. Stimuli are represented by competing subpopulations.

(A) Competition for representation of a mixture of 2 concurrent stimuli. Shown is the normalized average activity of two sub-populations, in response to mixtures of the preferred stimuli of the two populations. For mixtures containing predominately one stimulus (mixture proportions close to 0 and 1), the populations are strongly in competition, and the network represents exclusively the stronger of the two stimuli (responses near 0 and 1). For intermediate mixture proportions, competition causes a rapid shift between representations of the two stimuli (deviation from diagonal reference line). (B) Increasing the gain of the network (black line: 1.3, blue: 1.5, red: 1.8) increases the stability of representations, and increases the rate of switching between representations due to stronger competition.

Figure 9. Excitatory neurons are strongly tuned; inhibitory neurons are poorly tuned.

Tuning properties of excitatory and inhibitory neurons. (A) Representative tuning curves for 3 excitatory (red, 1-3) and 3 inhibitory (blue, 4-6) neurons in a WTA network after the learning process. Excitatory neurons exhibit strong and narrowly tuned preference for certain inputs, in contrast to inhibitory neurons. (B) Distribution of the orientation selectivity index (OSI) across all excitatory and inhibitory neurons in a WTA network, demonstrating the discrepancy of tuning on a population level. (C) Simulation of the same learning rule for synapses onto excitatory as well as inhibitory neurons yields orientation-tuned neurons in both populations.

Figure 10. Inhibition of excitatory neurons.

Excitatory neurons are predominantly inhibited by subsets of the inhibitory neurons that project to them. (A) Representative examples of the inhibition to excitatory neurons in a learned WTA network, during presentation of a stimulus. The vertical axis indicates the percentage of the total inhibitory REE (see definition of REE in text) that an individual inhibitory neuron delivers to this particular excitatory neuron. Few (usually 2 or 3) inhibitory neurons provide the major part of the inhibition. (B) Histogram of all the REE contributions (in %) from inhibitory neurons, across all excitatory neurons in the WTA network. The distribution shows that few inhibitory neurons provide the major part of the inhibitory REE on an excitatory neuron. This specialization is a result of the BCM learning rule, which is followed also by inhibitory synapses onto excitatory neurons.