Optimal Interplay between Synaptic Strengths and Network Structure Enhances Activity Fluctuations and Information Propagation in Hierarchical Modular Networks

Abstract

In network models of spiking neurons, the joint impact of network structure and synaptic parameters on activity propagation is still an open problem. Here, we use an information-theoretical approach to investigate activity propagation in spiking networks with a hierarchical modular topology. We observe that optimized pairwise information propagation emerges due to the increase of either (i) the global synaptic strength parameter or (ii) the number of modules in the network, while the network size remains constant. At the population level, information propagation of activity among adjacent modules is enhanced as the number of modules increases until a maximum value is reached and then decreases, showing that there is an optimal interplay between synaptic strength and modularity for population information flow. This is in contrast to information propagation evaluated among pairs of neurons, which attains maximum value at the maximum values of these two parameter ranges. By examining the network behavior under the increase of synaptic strength and the number of modules, we find that these increases are associated with two different effects: (i) the increase of autocorrelations among individual neurons and (ii) the increase of cross-correlations among pairs of neurons. The second effect is associated with better information propagation in the network. Our results suggest roles that link topological features and synaptic strength levels to the transmission of information in cortical networks.

Keywords: cortical network models; delayed transfer entropy; hierarchical modular networks; neural activity fluctuations; neural information processing.

Figure 1

Examples of hierarchical modular networks of different hierarchical levels. (Upper row) Schematic representation of the network for $H=$ 0, 2, and 3. In the figures, only networks with $N=2^{11}$ and exclusively excitatory neurons were used for the ease of visualization and to highlight the intermodular connections. (Bottom row) Adjacency matrices for networks with $N=2^{13}$ neurons (excitatory and inhibitory in the 4:1 ratio) and the same H levels as in the top row. Each dot represents a connection from a presynaptic neuron to a postsynaptic neuron. Blue dots represent excitatory neurons, and red dots represent inhibitory neurons. For each hierarchical level H, the module numbers are shown below the corresponding adjacency matrix.

Figure 2

Method to measure the delayed transfer entropy using the joint probability distributions. (a) First, we take two spike-trains of a pair of neurons in the network. (b) Then, we apply a delay d in one of them to determine the joint probability distributions $p(x_t,y_t)$ (indicated by the green arrow), $p(x_{t+1+d},x_{t+d},y_t)$ (indicated by the red arrow), and $p(y_{t+1},y_t)$ (indicated by the blue arrow). Next, we estimate the transfer entropy by inserting these distributions into Equation (10). (c) Example plots of $T{E}_{y\to x}$ and $T{E}_{x\to y}$ for a simple system of two coupled neurons (shown in the inset) with $x\to y$ connection delay ${\delta }_{x\to y}=2$ and $y\to x$ connection delay ${\delta }_{y\to x}=3$. The respective $TE$s are maximized when the measure delay d is the same as the corresponding connection delay.

Figure 3

Raster plot and activity plot of the network for selected values of J and H. For visibility, raster plots show spike times for a sample of only 2560 neurons, but the activity plots refer to all neurons in the network. Each column corresponds to a hierarchical level (from left to right: $H=0$, $H=7$, $H=9$), and each row corresponds to a synaptic strength ((upper row) $J=0.2$ mV; (bottom row) $J=0.8$ mV). In the case of modular networks ($H=7$ and $H=9$), spikes of neurons in the same module are indicated by the same color (black or gray), which alternates from one module to another to ease visualization. Although modules in the network with $H=9$ have a smaller number of neurons than modules in the network with $H=7$, the same number of neurons per module was chosen for the cases of $H=7$ and $H=9$ to allow a comparison.

Figure 4

Increases of J and H cause amplification of slow fluctuations and enhance information transfer. (a) Spike-train power spectra computed for $J=0.2$ mV and different values of H (indicated by different colors in the plot). (b) Same plot as in (a), but with $J=0.8$ mV. (c–e) Firing rate $\nu$, Fano factor $FF$, and correlation time ${\tau }_{\mathrm{c}}$ for different values of J (H values indicated by the same colors as in (a,b). (f) Average transfer entropy (computed as in Equation (11)) in a two-dimensional diagram where the abscissa represents synaptic strength J and the ordinate represents hierarchical level H. Values of $\langle TE\rangle$ are indicated by the color bar to the right side.

Figure 5

Spike-train autocorrelation $c_{xx}\left(\tau \right)$ and cross-correlation $c_{xy}\left(\tau \right)$ for selected pairs of parameters (H,J). Left: $c_{xx}$. Right: $c_{xy}$. The selected pairs (J,H) correspond to all possible combinations taken from the sets $J=\{0.2,0.4,0.6,0.8\}$ and $H=\{0,2,4,6,8\}$. For better visualization, $c_{xx}$ and $c_{xy}$ for the pairs (J,H) are plotted over the plot of $\langle TE\rangle$ in the J-H diagram. The $c_{xx}$ is extracted from $K=$ 10,000 randomly chosen neurons and the $c_{xy}$ from $K=$ 10,000 randomly chosen pairs of neurons.

Figure 6

Relation of connectivity and slow fluctuations. (a) Values of connectivity inside a module (${ϵ}_{\mathrm{in}}$) as H increases (cf. Equations (14)–(18)). (b) Spike-train power spectra extracted for a small network with $N=2^{14}$ and $H=0$ for different values of $ϵ$.

Figure 7

Transfer entropy and mutual information among modules. (a) Transfer entropy evaluated among modules $\langle T{E}^{\left(H\right)}\rangle$ in the two-dimensional diagram where the ordinate represents the hierarchical level H and the abscissa represents the synaptic strength J. Inset: boxplots of $\langle T{E}^{\left(H\right)}\rangle$ for fixed values of H. (b) Mutual information among modules $\langle M{I}^{\left(H\right)}\rangle$ in the same J-H diagram.