Decoding collective communications using information theory tools

Abstract

Organisms have evolved sensory mechanisms to extract pertinent information from their environment, enabling them to assess their situation and act accordingly. For social organisms travelling in groups, like the fish in a school or the birds in a flock, sharing information can further improve their situational awareness and reaction times. Data on the benefits and costs of social coordination, however, have largely allowed our understanding of why collective behaviours have evolved to outpace our mechanistic knowledge of how they arise. Recent studies have begun to correct this imbalance through fine-scale analyses of group movement data. One approach that has received renewed attention is the use of information theoretic (IT) tools like mutual information, transfer entropy and causation entropy, which can help identify causal interactions in the type of complex, dynamical patterns often on display when organisms act collectively. Yet, there is a communications gap between studies focused on the ecological constraints and solutions of collective action with those demonstrating the promise of IT tools in this arena. We attempt to bridge this divide through a series of ecologically motivated examples designed to illustrate the benefits and challenges of using IT tools to extract deeper insights into the interaction patterns governing group-level dynamics. We summarize some of the approaches taken thus far to circumvent existing challenges in this area and we conclude with an optimistic, yet cautionary perspective.

Keywords: causation entropy; collective behaviour; mutual information; transfer entropy.

Figure 1.

Shannon entropy. (a) The Shannon entropy in nats for the simplest case of only two possible outcomes, p and 1 − p [33]. (b,c) Illustration of how the Shannon entropy of a standard movement variable, individual orientation, can vary with ecological context. A group of animals can show greater variability in their orientations as they forage semi-independently of one another in an area (b), but this variability can drop as they travel together towards a common goal (c). On average, the change in variability between (b) and (c) corresponds to a drop in Shannon entropy, in nats, from (b) H(X) = 2.04 ± 0.006 to (c) H(X) = 1.32 ± 0.03. Estimates of p(x) were calculated from 1000 random samples of X, drawn either from a uniform distribution bounded by [0, 2π] or a Gaussian one (mean = 0, sd = 0.25) modulo 2π. Values of x were then sine transformed and binned to estimate p(x). Bin widths were defined by the optimal width determined in the uniform distribution using the Freedman–Diaconis algorithm [52]. We then replicated this process 1000 times to estimate the mean and standard deviation of H(X) for each distribution.

Figure 2.

Ant model trajectory. A sample trajectory of the ant model used as an example throughout this section. The position of each of the three ants is given on the abscissa, and each horizontal slice shows the configuration of the system at different times (both measured in arbitrary model units). Each dashed line shows what the trajectory of the like-coloured ant would be in the absence of any stochasticity. Note how the middle ant (green) initially takes a step backwards, which results in its follower (blue) making a backwards move one time step later.

Figure 3.

Covariance versus mutual information. (a) cov(X_n(t), X_n(t − 1)) for the ant model with n = 5 (red), 10 (green) and 20 (blue). The bottom dashed curve is the covariance for n = 0 (the lead ant), which serves as a lower bound. The upper dashed curve is the common curve that each covariance obeys for 0 < t ≤ n. (b) MI(X_n(t); X_n(t − 1)) in units of bits for the same set of values of n. The order of the curves here is reversed, with the n = 0 curve now serving as an upper bound. The inset of (a) shows that the same ordering can be achieved with the covariance if it is normalized by the product of the standard deviations in X_n(t) and X_n(t − 1), but at the cost of compressing the curves to the unit interval.

Figure 4.

Transfer entropy between different pairs of ants. The transfer entropy ${\text{TE}}_{X_5\to X_n}(t)$ is plotted in units of bits for the ant model for n = 6 (red), 10 (green), 15 (blue) and 25 (purple). The transfer entropy in each case is zero for 0 < t ≤ n − 5 and then increases logarithmically until saturation occurs for t > n. (Note that the abscissa is on a log scale to make the logarithmic growth look linear.)

Figure 5.

Comparison of transfer and causation entropy. The long-time limiting value of the transfer entropy ${\text{TE}}_{X_{n-1}\to X_n}$ and the t > 1 value of the causation entropy ${\text{CSE}}_{X_{n-1}\to X_n}$ are plotted in red and blue, respectively, for the ant model as a function of the position behind the leader (n > 0) in units of bits. The constant CSE for each pair of adjacent ants is a reflection of the homogeneity of the model’s rule-based interactions. Any differences in the dynamics of each ant derive from the indirect influence of the ants further up front, which is reflected in the growth of the TE curve as that number increases. The logarithmic character of this growth suggests that the efficiency of indirect influence falls off with distance.

Figure 6.

Summary of IT metrics. (a) The three IT metrics calculated for the ant model are summarized here in standard Venn diagram format. Note how in going from mutual information to causation entropy, the uncertainty in the variable X_n(t) is successively reduced by conditioning it on more and more variables. (b) All the IT metrics presented in this section make comparisons between a distribution of one variable, generically p(X), and a second distribution of that variable conditioned on another, p(X|Y_i). The size of the corresponding IT metric depends upon how much that conditioning reduces the uncertainty in X, which, for the normally distributed random variables in the ant model, is directly related to how much conditioning reduces the width (variance) of the distribution. In the Venn diagram, this reduction corresponds with the extent to which the area of the circle representing X is reduced by its overlap with the circle representing Y_i.

Figure 7.

Collective gradient climbing strategies. (a) When attempting to climb an environmental gradient (intensifying from white to blue) individual errors can be cancelled out, and, by staying together, a group’s mean direction (dashed arrow) can lead up the gradient (the many-wrongs hypothesis) [66,67]. (b) When individuals in preferred locations slow down relative to those in lower quality regions (shown by shorter trailing wakes), social cohesion can combine with the resultant speed differential to rotate the group up the gradient and lead to a case of emergent sensing [68]. (c) In theory, we may also expect variable interaction strengths based on local conditions (line/arrow thickness). Individuals further up the gradient are less social as they take advantage of their location in the gradient, while individuals in less favourable locations show stronger social ties as they seek better locations along the gradient [69]. (d) If individuals are cooperating, those located up a gradient could also signal to others the quality of the resource based on a local scalar measure (circles) [70]. (e) The preceding examples may also apply at the cellular level. Cell clusters are polarized radially, with each cell producing a force in the direction of polarization, resulting in a net motion up the gradient similar to (a) [71].

Figure 8.

Probability distributions for each sensory scenario. Panels show the distribution of values for the order parameter (a), the mutual information (b), the transfer entropy (c) and the causation entropy (d). Curves are coloured by scenario (local interactions, black; global interactions, red). Data were collected from 100 replicate simulations, each of which was allowed to first reach a steady state. For each replicate, IT metrics were computed for each pair of agents, and the results were compiled into the plotted distributions. The Freedman–Diaconis rule [52] was used to histogram all the datasets. Simulation parameters: N = 50, L = 10, Δt = 1,v_o = 0.1, R = 1, 15 and η = 10^−3/2.

Figure 9.

Distribution of causal neighbours. Influential neighbours were identified using the oCSE algorithm (see electronic supplementary material, S1). Dashed lines represent the median value of each distribution (4 and 8 for the global and local scenarios, respectively).

Figure 10.

IT metrics: theory versus simulation. (a) The mutual information MI(X_n(t); X_n(t − 1)) plotted as a function of time. This quantity is independent of n for t < n, allowing for data points for multiple agents at the same time to be compiled. The black curve is the analytic result, and the red, green and blue curves correspond to datasets of 50, 450 and 950 data points, respectively. (b) The long-time transfer entropy ${\text{TE}}_{X_{n-1}\to X_n}$ plotted as a function of the ant number n. The black curve is again the analytic result, and the coloured curves represent datasets of the same sizes as in (a), except that in this case they are compiled over time points for fixed n since the TE is constant for times t > n.

Figure 11.

The effects of aggregating data across time points and agents. In the ant model, for a chain of five ants with a mean step size of five units, the step size ΔX_n(t) can be shown to sample, for different values of t and n, from five different normal distributions, plotted as dashed curves. The black solid curve represents the mean of these distributions and is what one samples if the data for ΔX_n(t) is aggregated across all values.