Revealing the Dynamics of Neural Information Processing with Multivariate Information Decomposition

Abstract

The varied cognitive abilities and rich adaptive behaviors enabled by the animal nervous system are often described in terms of information processing. This framing raises the issue of how biological neural circuits actually process information, and some of the most fundamental outstanding questions in neuroscience center on understanding the mechanisms of neural information processing. Classical information theory has long been understood to be a natural framework within which information processing can be understood, and recent advances in the field of multivariate information theory offer new insights into the structure of computation in complex systems. In this review, we provide an introduction to the conceptual and practical issues associated with using multivariate information theory to analyze information processing in neural circuits, as well as discussing recent empirical work in this vein. Specifically, we provide an accessible introduction to the partial information decomposition (PID) framework. PID reveals redundant, unique, and synergistic modes by which neurons integrate information from multiple sources. We focus particularly on the synergistic mode, which quantifies the "higher-order" information carried in the patterns of multiple inputs and is not reducible to input from any single source. Recent work in a variety of model systems has revealed that synergistic dynamics are ubiquitous in neural circuitry and show reliable structure-function relationships, emerging disproportionately in neuronal rich clubs, downstream of recurrent connectivity, and in the convergence of correlated activity. We draw on the existing literature on higher-order information dynamics in neuronal networks to illustrate the insights that have been gained by taking an information decomposition perspective on neural activity. Finally, we briefly discuss future promising directions for information decomposition approaches to neuroscience, such as work on behaving animals, multi-target generalizations of PID, and time-resolved local analyses.

Keywords: computation; cortical circuits; entropy; higher-order interactions; information theory; neural recording; neuroscience.

Figure 1

The total information contained in the activity of two source neurons $X^1$ and $X^2$ about the activity of a target neuron Y consists of multiple parts. This total information, $I(X^1,X^2;Y)$, is represented by the outermost oval. Contained within this total is the information that $X^1$ and $X^2$ each independently carry about Y, represented by the red circle for $I(X^1;Y)$ and the blue circle for $I(X^2;Y)$. These independent sources can carry redundant information, represented by the overlapping striped section labeled $\mathrm{Red}(X^1,X^2;Y)$. The non-redundant information each source neuron accounts for is the unique information $\mathrm{Unq}(X^1;Y)$ and $\mathrm{Unq}(X^2;Y)$. Finally, and most relevantly for the study of information processing, the joint state of $X^1$ and $X^2$ can account for the activity of Y to some degree. This information is not accounted for by either source independently and is the synergistic information that $X^1$ and $X^2$ carry about Y, i.e., $\mathrm{Syn}(X^1,X^2;Y)$. The purple space making up the difference between what is included in the $I(X^1;Y)$ and $I(X^2;Y)$ circles and the total information $I(X^1,X^2;Y)$ represents $\mathrm{Syn}(X^1,X^2;Y)$.

Figure 2

Partial information lattices. On the left is the lattice for two predictor variables, and on the right is the lattice for three predictor variables. Each lattice is constructed and annotated following the notation in [1]. Lattice vertices are partial information atoms, i.e., the unique modes of information-sharing that comprise the overall joint mutual information. Information atoms are denoted by index only: for example, $\left\{1\right\}\left\{2\right\}$ is the information redundantly disclosed by $X^1$ or $X^2$, $\left\{1\right\}\left\{23\right\}$ is the information disclosed by $X^1$ or ($X^2$ and $X^3$), etc. Lattice edges indicate which atoms subsume other atoms. Atoms connected to and below other atoms consist of components of and/or subsets of the higher atoms, for example, $\left\{1\right\}\left\{2\right\}⪯\left\{1\right\}\left\{23\right\}$ and $\left\{2\right\}\left\{13\right\}$, since information disclosed by $\left\{1\right\}\left\{2\right\}$ would also be visible to $\left\{1\right\}\left\{23\right\}$ and $\left\{2\right\}\left\{13\right\}$ if we did not use the Mobius inversion.

Figure 3

Example of how neuronal activity recordings can be subjected to PID-based study. Rapid extraction and sectioning of mouse brains created 400 $\mathsf{\mu }$m sagittal slices of cortex. Incubation in nutrient media allowed the slice to culture, restoring organotypic connectivity patterns [47]. Placing the culture on a high-density 512-channel microelectrode array allowed for recordings permitting spike sorting via PCA waveform analysis, to attribute observed spikes to individual neurons. The resulting spike rasters reflect the millisecond-level precision activation patterns of hundreds of neurons. Analyzing the effective connectivity among neurons using transfer entropy enables extraction of the full effective network upon which the spiking dynamics occurred. The computational triads, consisting of two neurons (source neurons) connecting to a common third neuron (target neuron), can then be identified. PID can then be applied separately to each computational triad. Variations across triads can be used to analyze how synergy (or redundancy) varies as a function of network properties such as the boundaries of a rich club or as a function of the number of feedback and recurrent connections. Figure adapted from [2].

Figure 4

Empirical study of circuit dynamics using PID can reveal covariates of synergistic integration. (A) Synergistic integration is robustly positively correlated with the strength of feedforward connections in a triad. Left: A single representative network with 3000+ computational triads shows that feedforward connection strength (i.e., triad TE) was positively related to the synergy for each triad. Right: This positive relationship was reliably observed over 75 different effective networks analyzed from 25 different cultures. Images from [2]. (B) The mean synergy across all triads localized to rich clubs was significantly higher than the synergy in triads not localized to a rich club. The inset ‘$\ast \ast \ast \ast \ast$’ indicates $p<1\times {10}^{-9}$ for the difference between conditions. Image from [2]. (C) Sorting computational triads based on the number of connections from the target neuron back to the source neurons (feedback connections) and connections between the source neurons (recurrent connections) revealed more synergy for triads with more recurrent connections. Image from [4]. (D) Feedforward and recurrent connections were both positively related to synergistic integration, while feedback connections were not reliably related to synergistic integration. Image from [4]. (E) Synergy (normalized to be the proportion of the receiver entropy, or pH${}^{rec}$) was non-monotonically related to the similarity of the spiking of the source neurons (normalized to be the proportion of the maximum possible mutual information, or pMI${}^{max}$) when analyzed across a wide range of timescales. Each panel shows the results for a single timescale. Note that the x-axis is logarithmically scaled, and the range varies across panels. The timescale varies from left to right across panels, ranging from short (e.g., 3 ms, 5 ms, and 11 ms) to long (e.g., 485 ms, 1044 ms, and 2250 ms), as indicated above each panel. At short timescales, the maximum similarity in spiking between source neurons was low, and both synergy (green solid line) and redundancy (purple dashed line) were positively related to the similarity of the spiking between source neurons. At long timescales, the total similarity was high, and only redundancy was positively related to the similarity of the source neurons. Synergy was maximized when the source neurons were intermediately similar, with pMI${}^{max}=0.07$, as marked with a vertical dashed line in each panel. Image related to the similarity from [3].