Comparison of derivative-based and correlation-based methods to estimate effective connectivity in neural networks

Abstract

Inferring and understanding the underlying connectivity structure of a system solely from the observed activity of its constituent components is a challenge in many areas of science. In neuroscience, techniques for estimating connectivity are paramount when attempting to understand the network structure of neural systems from their recorded activity patterns. To date, no universally accepted method exists for the inference of effective connectivity, which describes how the activity of a neural node mechanistically affects the activity of other nodes. Here, focussing on purely excitatory networks of small to intermediate size and continuous node dynamics, we provide a systematic comparison of different approaches for estimating effective connectivity. Starting with the Hopf neuron model in conjunction with known ground truth structural connectivity, we reconstruct the system's connectivity matrix using a variety of algorithms. We show that, in sparse non-linear networks with delays, combining a lagged-cross-correlation (LCC) approach with a recently published derivative-based covariance analysis method provides the most reliable estimation of the known ground truth connectivity matrix. We outline how the parameters of the Hopf model, including those controlling the bifurcation, noise, and delay distribution, affect this result. We also show that in linear networks, LCC has comparable performance to a method based on transfer entropy, at a drastically lower computational cost. We highlight that LCC works best for small sparse networks, and show how performance decreases in larger and less sparse networks. Applying the method to linear dynamics without time delays, we find that it does not outperform derivative-based methods. We comment on this finding in light of recent theoretical results for such systems. Employing the Hopf model, we then use the estimated structural connectivity matrix as the basis for a forward simulation of the system dynamics, in order to recreate the observed node activity patterns. We show that, under certain conditions, the best method, LCC, results in higher trace-to-trace correlations than derivative-based methods for sparse noise-driven systems. Finally, we apply the LCC method to empirical biological data. Choosing a suitable threshold for binarization, we reconstruct the structural connectivity of a subset of the nervous system of the nematode C. elegans. We show that the computationally simple LCC method performs better than another recently published, computationally more expensive reservoir computing-based method. We apply different methods to this dataset and find that they all lead to similar performances. Our results show that a comparatively simple method can be used to reliably estimate directed effective connectivity in sparse neural systems in the presence of spatio-temporal delays and noise. We provide concrete suggestions for the estimation of effective connectivity in a scenario common in biological research, where only neuronal activity of a small set of neurons, but not connectivity or single-neuron and synapse dynamics, are known.

Keywords: C. elegans; Computational connectomics; Dynamical systems; Effective connectivity; Hopf model; Ornstein–Uhlenbeck process; Structural connectivity.

Fig. 1

Comparison of GT and EStC connectivity matrices in a Hopf Model using the thresholded LCC method, connection probability p = 0.1. Panel A shows the original GT connectivity matrix used to generate time-series data with a Hopf model. Panel B shows the EStC matrix using the LCC method (Materials and Methods, Lagged correlation). Nearly perfect agreement with the GT matrix is observed. The Pearson correlation coefficient is 0.97 between the matrices. Panel C shows the EStC matrix using the DDC method. This method finds some of the right connections, but the connection strength is off-target, and there are many false positives. This leads to a lower correlation (0.52) between the EStC matrix computed with DDC and the GT connectivity matrix (panel A). Due to rounding to one decimal place, 2 values are not shown in the GT connectivity matrix, this leads to 7 (instead of 9) visible connections in the GT connectivity matrix. Panels D-F show the corresponding network graphs for the GT connectivity (Panel D) and the two estimated graphs for LCC (Panel E) and DDC (Panel F). Two connections in Panel D are hardly visible because of their small numerical value.

Fig. 2

Performance of different methods for N=10 for the Hopf model. Pearson correlation coefficients between EStC matrices estimated using different methods and the GT matrix of the given model parameters for a Hopf model. The number of neurons is set to N=10, while the connection probability p increases from 0.1 to 1.0. For small p (p≤0.5), LCC in conjunction with the DDC algorithm yielded the highest correlation out of all the methods used (row 7). At higher values of p correlation-based methods (rows 1 and 2) and cut LCC (row 6) perform best. LCC and thresholded LCC (rows 4 and 5) perform equally well. All LCC and correlation-based methods perform better than DDC-based methods (rows 8 and 9). NTE and Partial Correlation (rows 10 and 3) show fewer changes in performance as p is changed, and perform worse than LCC- based methods at low p and equally well at higher p, where they also perform better than DDC-based methods. The value per cell is calculated by taking the mean of the correlation in 100 simulations rounded to 2 decimal places.

Fig. 3

Performance of different methods for N=100 for the Hopf model. Pearson correlation coefficients between EStC matrices estimated using different methods and the GT matrix of the given model parameters for a Hopf model. The number of neurons is set to N=100, while the connection probability p increases from 0.01 to 0.1.

Fig. 4

Performance of different methods for N=250 for the Hopf model. Pearson correlation coefficients between EStC matrices estimated using different methods and the GT matrix of the given model parameters for a Hopf model. The number of neurons is set to p, while the connection probability N=250 increases from 0.004 to 0.04.

Fig. 5

Performance of different models for N=10 for the linear model. Figure layout similar to Fig. 2, but for the linear neuron model (Eq.8).

Fig. 6

Performance of different models for N=100 for the linear model. Figure layout similar to Fig.2, but for the linear neuron model (Eq. 8).

Fig. 7

Performance of different models for N=250 for the linear model. Figure layout similar to Fig. 2, but for the linear neuron model (Eq.8).

Fig. 8

Comparison of the simulated data using the GT, LCC- estimated connectivity and DDC-estimated connectivity for a network of 10 neurons with p = 0.1. Orange: Original simulation. Blue: Regeneration of the dynamics using LCC to obtain EC. Green: Regeneration of the dynamics using DDC to obtain EC. Panel A: Results for neuron 1. Panel B: Results for neuron 6. The real part of the corresponding Stuart-Landau equation (Eqs. 6 and 7) is here plotted as activity. The same random seed for the stochastic input to each node’s internal dynamics was used in original and both regenerated trajectories.

Fig. 9

Performance analysis of LCC method binarization thresholds on true and false positive connections in C. elegans time series data.The figure depicts the relationship between the cut threshold, seen on the x-axis and the corresponding counts of TPs and FPs. A threshold value of 0.2 (black dashed line) was chosen to have comparability against the methods proposed by Banerjee et al..

Fig. 10

Comparison of the GT and EStC connectivity for the 8 most active motor neurons of C. elegans. Panel A shows the EStC matrix using the LCC method with a binarization: values above a threshold of 0.2 are clipped to 1 and values below it to 0. Panel B shows the GT connectivity matrix for the 8 most active motor neurons of C. elegans. Comparing the matrices shows a TP count of 21 and an FP count of 4 as indicated in Fig. 9 by the black dashed line.

Fig. 11

Guide for method selection to estimate connectivity from activity in neural networks. The input to all methods is recorded GT activity data, for which assumptions on the presence or absence of delays and underlying dynamical model need to be made. Also, an initial estimate about the network size and its density should be made. With these inputs and assumptions, we can proceed with a choice of estimation method. For non-linear dynamics and small delays, LCC is the preferred method, whereas DDC is the preferred method for linear dynamics with small/ absent or large delays. For non-linear dynamics with large delays, the thresholded DDC method is preferred.