Synchronization and Inter-Layer Interactions of Noise-Driven Neural Networks

Abstract

In this study, we used the Hodgkin-Huxley (HH) model of neurons to investigate the phase diagram of a developing single-layer neural network and that of a network consisting of two weakly coupled neural layers. These networks are noise driven and learn through the spike-timing-dependent plasticity (STDP) or the inverse STDP rules. We described how these networks transited from a non-synchronous background activity state (BAS) to a synchronous firing state (SFS) by varying the network connectivity and the learning efficacy. In particular, we studied the interaction between a SFS layer and a BAS layer, and investigated how synchronous firing dynamics was induced in the BAS layer. We further investigated the effect of the inter-layer interaction on a BAS to SFS repair mechanism by considering three types of neuron positioning (random, grid, and lognormal distributions) and two types of inter-layer connections (random and preferential connections). Among these scenarios, we concluded that the repair mechanism has the largest effect for a network with the lognormal neuron positioning and the preferential inter-layer connections.

Keywords: biological neural networks; computer simulation; developing neural networks; inter-layer interactions; noise-driven synchronization; repair mechanism of neural networks; spike-timing-dependent plasticity; synchronous firing.

Figure 1

Schematic illustration of the topology of a coupled network consisting of a high connectivity layer (top) and a low connectivity layer (bottom). The arrows show the direction of inter-layer synaptic connections.

Figure 2

The threshold value ($N_c^{*}$) of network connections to induce synchronous firing as a function of the standard deviation (D) of a Gaussian white noise. The dotted curve is a fit of the data points, which suggests no synchronization for D ≤ 18.5 μA/cm². Data were calculated by averaging 30 realizations and using the following set of parameters: A₊ = 0.013 and M = 50.

Figure 3

Phase diagram of a developing neural network, which consists of a background activity state (BAS), a transition state (TS), and a synchronous firing state (SFS). The inset shows the synchronization order parameter <Ψ_s> at various values of network connectivity. Data were calculated by averaging 30 realizations and using the following set of parameters: A₊ = 0.013, D = 25 μA/cm², and M = 50.

Figure 4

The synchronization order parameter as a function of the average number of connections per neuron (N_c/M) for M = 50, 80, 100, and 120. Data were calculated by averaging 30 realizations and using the following set of parameters: A₊ = 0.013 and D = 25 μA/cm².

Figure 5

Synchronization order parameter of a BAS layer (layer 2, N_c2 = 1000 and A₊ = 0.010) as a function of simulation time for various values of N_i after its momentary coupling to a SFS layer (layer 1, N_c1 = 1000 and A₊ = 0.013). The time average <Ψ_s2> was calculated by the average of 30 simulations with different seeds.

Figure 6

The time series of neuron firing and the average membrane potential of layer 2 before coupling (A), after coupling (B), and after disconnecting the coupling (C) to a SFS layer. Here we used the following set of parameters: N_c2 = 1000 and A₊ = 0.010 for layer 2, N_c1 = 1000 and A₊ = 0.013 for layer 1, and N_i = 180.

Figure 7

Phase diagram of layer 2 in a network of two coupled neural layers by varying N_i and N_c2. The phase diagram was plotted based on <Ψ_s>, which was calculated by the average of 30 simulations with different seeds. Here we used the following set of parameters: N_c1 = 1000, and A₊ = 0.013 for both layers.

Figure 8

The synchronization order parameter of a BAS layer as a function of N_c in a single-layer noisy network (A), or as a function of N_i in a coupled network consisting of a BAS layer (N_c2 = 300) and a SFS layer (N_c1 = 1000) (B). Both the STDP (A₊ = 0.013) and the inverse STDP (A₋ = 0.013) learning rules were considered. The synchronization order parameter was calculated by its average of 30 simulations with different seeds.

Figure 9

Phase difference of two coupled SFS layers as a function of N_i. Before coupling, these two layers fired at the same frequency but with a phase difference of 2.9 radians. Data were calculated by averaging 5 realizations and using the following set of parameters: N_c1 = N_c2 = 1000 and A₊ = 0.013 for both layers.

Figure 10

Three types of neuron positioning on a layer, including a random distribution (A), a grid distribution (B), and a lognormal distribution (C). For all three cases, N_c = 1000. In (D), we showed the distribution of neurons' degree of connections for the three types of neuron positioning in (A–C), each of which was calculated by averaging an ensemble of 1000 seeds.

Figure 11

Synchronization order parameter of layer 2 as a function of N_i. Four scenarios of different inter-layer connection mechanisms and learning rules were displayed, including a network with the random connection and the STDP learning, one with the preferential connection and the STDP, one with the random connection and the inverse STDP, and the other with the preferential connection and the inverse STDP. Here we used the following set of parameters: N_c1 = 1000, N_c2 = 300, and A₊ = 0.013 for both layers. The time average <Ψ_s2> was calculated by the average of 30 simulations with different seeds.

Figure 12

Synchronization order parameter of layer 2 as a function of N_i to demonstrate our best and worst designs of two coupled neural layers in synchronization enhancement. Both the STDP and the inverse STDP learning rules were considered. Here we used the following set of parameters: N_c1 = 1000, N_c2 = 300, and A₊ = 0.013 for both layers. The time average <Ψ_s2> was calculated by the average of 30 simulations with different seeds.