Mutual information rate and bounds for it

Abstract

The amount of information exchanged per unit of time between two nodes in a dynamical network or between two data sets is a powerful concept for analysing complex systems. This quantity, known as the mutual information rate (MIR), is calculated from the mutual information, which is rigorously defined only for random systems. Moreover, the definition of mutual information is based on probabilities of significant events. This work offers a simple alternative way to calculate the MIR in dynamical (deterministic) networks or between two time series (not fully deterministic), and to calculate its upper and lower bounds without having to calculate probabilities, but rather in terms of well known and well defined quantities in dynamical systems. As possible applications of our bounds, we study the relationship between synchronisation and the exchange of information in a system of two coupled maps and in experimental networks of coupled oscillators.

Figure 1. Results for two coupled maps. [Eq. (3)] as (green online) filled circles, [Eq. (5)] as the (red online) thick line, and [Eq. (7)] as the (blue online) crosses.

In (A) and in (B) . The units of , , and are [bits/iteration].

Figure 2. Black filled circles represent a Chua’s circuit and the numbers identify each circuit in the networks.

Coupling is diffusive. We consider 4 topologies: 2 coupled Chua’s circuit (A), an array of 3 coupled circuits, an array of 4 coupled circuits, and a ring formed by 4 coupled circuits.

Figure 3. Results for experimental networks of Double-Scroll circuits.

On the left-side upper corner pictograms represent how the circuits (filled circles) are bidirectionally coupled. as (green online) filled circles, as the (red online) thick line, and as the (blue online) squares, for a varying coupling resistance . The unit of these quantities shown in these figures is (kbits/s). (A) Topology I, (B) Topology II, (C) Topology III, and (D) Topology IV. In all figures, increases smoothly from 1.25 to 1.95 as varies from 0.1k to 5k. The line on the top of the figure represents the interval of resistance values responsible to induce almost synchronisation (AS) and phase synchronisation (PS).

Figure 4. This picture is a hand-made illustration.

Squares are filled as to create an image of a stochastic process whose points spread according to the given Lyapunov exponents. (A) A small box representing a set of initial conditions. After one iteration of the system, the points that leave the initial box in (A) go to 4 boxes along the diagonal line [filled squares in (B)] and 8 boxes off-diagonal (along the transverse direction) [filled circles in (B)]. At the second iteration, the points occupy other neighbouring boxes as illustrated in (C) and after an interval of time the points do not spread any longer (D).