From the phenomenology to the mechanisms of consciousness: Integrated Information Theory 3.0

Abstract

This paper presents Integrated Information Theory (IIT) of consciousness 3.0, which incorporates several advances over previous formulations. IIT starts from phenomenological axioms: information says that each experience is specific--it is what it is by how it differs from alternative experiences; integration says that it is unified--irreducible to non-interdependent components; exclusion says that it has unique borders and a particular spatio-temporal grain. These axioms are formalized into postulates that prescribe how physical mechanisms, such as neurons or logic gates, must be configured to generate experience (phenomenology). The postulates are used to define intrinsic information as "differences that make a difference" within a system, and integrated information as information specified by a whole that cannot be reduced to that specified by its parts. By applying the postulates both at the level of individual mechanisms and at the level of systems of mechanisms, IIT arrives at an identity: an experience is a maximally irreducible conceptual structure (MICS, a constellation of concepts in qualia space), and the set of elements that generates it constitutes a complex. According to IIT, a MICS specifies the quality of an experience and integrated information ΦMax its quantity. From the theory follow several results, including: a system of mechanisms may condense into a major complex and non-overlapping minor complexes; the concepts that specify the quality of an experience are always about the complex itself and relate only indirectly to the external environment; anatomical connectivity influences complexes and associated MICS; a complex can generate a MICS even if its elements are inactive; simple systems can be minimally conscious; complicated systems can be unconscious; there can be true "zombies"--unconscious feed-forward systems that are functionally equivalent to conscious complexes.

Figure 1. Existence: Mechanisms in a state having causal power.

(A) The dotted circle indicates elements ABC as the candidate set of mechanisms. Elements outside the candidate set (D, E, F) are taken as background conditions (external constraints). The logic gates A, B, and C are represented as is customary in neural circuits rather than electronic circuits. The arrows indicate directed connections between the elements. (B) The set's mechanisms ABC determine the transition probability matrix (TPM) of the set under the background conditions of DEF (here DEF(t ₋₁) = DEF(t ₀) = 010). With element D fixed to D = 0, element A, for instance, receives inputs from B and C and outputs to B and C. The OR gate A is on (1) if either B, or C, or both were on at the last time step, and off (0) if BC was 00. Filled circles denote that the state of an element is ‘1’, open circles indicate that the state of an element is ‘0’. The current state of ABC is 100.

Figure 2. Composition: Higher order mechanisms can be composed by combining elementary mechanisms.

The set ABC has 3 elementary mechanisms A, B, and C (at the bottom). Second-order mechanisms AB, AC, and BC are shown in the middle row and the third-order mechanism ABC (corresponding to the full set) is shown at the top. Altogether, the figure indicates the power set of possible mechanisms in set ABC. In the figure, each mechanism is highlighted by a red shaded area. The current state of the elements inside the candidate set but outside of a mechanism is undetermined for the mechanism under consideration.

Figure 3. Information requires selectivity.

A mechanism generates information to the extent that it selectively constrains a system's past states. Element constrains the past states of depending on its mechanism (AND gate) and its current state. The constrained distribution of past states is called A's cause repertoire. (A) The connections between and are noisy. A's cause repertoire is thus unselective, since could have followed from any state of with equal probability. (B) In the case of deterministic connections and current state , A's cause repertoire is maximally selective, because all states except are ruled out as possible causes of . (C) In the case of deterministic connections and current state , A's cause repertoire is much less selective than for , because only state is ruled out as a possible cause of .

Figure 4. Information: “Differences that make a difference to a system from its own intrinsic perspective.”

A mechanism generates information by constraining the system's past and future states. (Top) The candidate set consisting of OR, AND, and XOR gates is shown in its current state 100. We consider the purview of mechanism , highlighted in red, over the set in the past (blue) and in the future (green). (Bottom center) The same network is displayed unfolded over three time steps, from (past), (current) to (future). Gray-filled circles are undetermined states. The current state of mechanism A constrains the possible past and future system states compared to the unconstrained past and future distributions . For example, rules out the two states where as potential causes. The constrained distribution of past states is A's cause repertoire (left). The constrained distribution of future states is A's effect repertoire (right). Cause information (ci) is quantified by measuring the distance D between the cause repertoire and the unconstrained past repertoire ; effect information (ei) is quantified by measuring the distance D between the effect repertoire and the unconstrained future repertoire . Note that the unconstrained future repertoire is not simply the uniform distribution, but corresponds to the distribution of future system states with unconstrained inputs to each element. Cause-effect information (cei) is then defined as the minimum of ci and ei.

Figure 5. A mechanism generates information only if it has both selective causes and selective effects within the system.

(A) Element A receives input from the system and specifies a selective cause repertoire. However, since it has no outputs to the system it does not specify a selective effect repertoire. (B) Element A receives no input from the system and therefore it does not specify a selective cause repertoire. In both cases the cause-effect information cei generated by mechanism A is zero (the minimum between cause and effect information).

Figure 6. Integrated information: The information generated by the whole that is irreducible to the information generated by its parts.

Integrated information is quantified by measuring the distance between the cause repertoire specified by the whole mechanism and the partitioned mechanism (the same for the effect repertoire). MIP is the minimum information partition – the partition of the mechanism that makes the least difference to the cause and effect repertoires (indicated by dashed lines in the unfolded system). Partitions are performed by noising connections between the parts (those that cross the dashed lines, see Text S2).

Figure 7. A mechanism generates integrated information only if it has both integrated causes and integrated effects.

(A) The mechanisms of element A and B are independent, having separate causes and effects. From the intrinsic perspective of the system, the joint mechanism AB does not exist, since it can be partitioned (red dashed line) without making any difference to the system. (B) The mechanism AB generates integrated information both in the past and in the future. Since it cannot be partitioned without loss, it exists intrinsically. (C) The mechanism AB generates integrated information in the past but not in the future. (D) The mechanism AB generates integrated information in the future but not in the past. In both cases, the joint mechanism does not exist intrinsically.

Figure 8. The maximally integrated cause repertoire over the power set of purviews is the “core cause” specified by a mechanism.

All purviews of mechanism BC for the past are considered. Only the purview that generates the maximal value of integrated information, φ ^Max, exists intrinsically as the core cause of the mechanism (or effect when considering the future). In this case, the core cause is .

Figure 9. A concept: A mechanism that specifies a maximally irreducible cause-effect repertoire.

The core cause and effect of mechanism A are and , respectively. Together, they specify “what” the concept of A is about. The value of the concept specifies “how much” the concept exists intrinsically.

Figure 10. Information: A conceptual structure C (constellation of concepts) is the set of all concepts generated by a set of elements in a state.

(A) The candidate set ABC – a system composed of mechanisms in a state. (B) The power set of ABC's mechanisms. (C) The concepts generated by the candidate set. Core causes are plotted on the left, core effects on the right. values are shown in blue fonts in the middle of the cause and effect repertoires of each mechanism. Note that all mechanisms in the power set are concepts, with the exception of mechanism AC, which can be fully reduced . (D) The concepts generated by the candidate set plotted in concept space, where each axis corresponds to a possible state of ABC. For ease of representation past and future subspaces are plotted separately, with only three axes each. The “null” concept p^uc is indicated by the small black crosses in concept space.

Figure 11. Assessing the conceptual information CI of a conceptual structure (constellation of concepts).

CI is quantified by measuring the distance in concept space between C, the constellation of concepts generated by a set of elements, and , the unconstrained past and future repertoire, which can be termed the “null” concept (in the absence of a mechanism, every state is equally likely). This can be done using an extended version of the earth mover's distance (EMD) that corresponds to the sum of the standard EMD for distributions between the cause-effect repertoires of all concepts and , weighted by their values. (A) Therefore, a system with many different elementary and higher order concepts has high CI, as shown here for the candidate set ABC. (B) By contrast, a system comprised of a single mechanism can only have one concept and thus has low CI.

Figure 12. Assessing the integrated conceptual information Φ of a constellation C.

Φ (“big phi”) is quantified by measuring the distance C between the constellation of concepts of the whole set of elements C and that of the partitioned set , using an extended version of the earth mover's distance (EMD). The set is partitioned unidirectionally (see text for the motivation) until the partition is found that yields the least difference between the constellations (MIP, the minimum information i.e. minimum difference partition). In this case, the MIP corresponds to “noising” the connections from AB to C. This partition leaves 2 concepts intact (A and B, with zero distance to A and B from constellation C, indicated by the red stars), while the other concepts are destroyed by the partition (gray stars). The distance between the whole and partitioned constellations thus amounts to the sum of the EMD between the cause-effect repertoires of the destroyed concepts and the “null” concept , weighted by their values (see Text S2).

Figure 13. A set of elements generates integrated conceptual information Φ only if each subset has both causes and effects in the rest of the set.

(A) A set of 6 elements is composed of two subsets that are not interconnected. The set reduces to 2 independent subsets of 3 elements each that can be partitioned without loss (dashed red line). The 6 element set does not exist intrinsically (dashed black oval). (B) All subsets of the 6 node set have causes and effects in the rest of the set. The 6 node set generates an integrated conceptual structure since it cannot be unidirectionally partitioned without loss of conceptual information. (C,D) A set of 6 elements divides into 2 subsets of 3 elements that are connected unidirectionally. (C) The left subset has causes in the rest of the set, but no effects. (D) The left subset has effects on the rest of the set, but no causes. In both cases, the set reduces to 2 subsystems of 3 elements each that can be unidirectionally partitioned without loss (dashed red line with directional arrow). The 6 element set does not exist intrinsically.

Figure 14. A complex: A local maximum of integrated conceptual information Φ.

Integrated conceptual information Φ is computed for the power set of elements of system ABCDEF (all possible candidate sets). By the exclusion postulate, among overlapping candidate sets, only one set of elements forms a complex, the one that generates the maximum amount of integrated conceptual information Φ^Max. In the example system the set of elements ABC form the complex. Therefore, no subset or superset of ABC can form another complex. Note that all candidate sets that include D, E, or F are not strongly integrated and thus have Φ = 0 (only one example is shown).

Figure 15. A quale: The maximally irreducible conceptual structure (MICS) generated by a complex.

An experience is identical with the constellation of concepts specified by the mechanisms of the complex. The Φ^Max value of the complex corresponds to the quantity of the experience, the “shape” of the constellation of concepts in qualia space completely specifies the quality of a particular experience and distinguishes it from other experiences.

Figure 16. A system can condense into a major complex and minor complexes that may or may not interact with it.

The set of elements ABC specifies the local maximum of integrated information Φ^Max and thus forms the major complex of the system. The sets of elements DE and FG also specify local maxima of integrated information albeit with lower Φ^Max than the main complex. DE and FG thus form minor complexes. The set of elements ABCDE is strongly integrated, but is excluded from forming a complex, since it overlaps with ABC, which is a local maximum of integrated information. The elements I, J, and L cannot be part of any complex since they do not have both causes and effects in the rest of the system. Neither can H and K, since they are part of a strictly feed-forward chain.

Figure 17. Qualia generated by modular, homogeneous and specialized networks.

(A) The modular network decomposes into three small complexes and their residual interactions. (B) The homogenous system forms a complex, but it has low Φ^Max and only 5 identical concepts. (C) The specialized network also forms a complex, with all but one concepts of its power set and a high Φ^Max value. In the middle row, the respective concepts of each system are listed. The bottom row shows the constellation of the respective complexes in qualia space (projected into 3 dimensions for the past and the future subspaces).

Figure 18. Quale generated by an inactive system.

Neural activity is not necessary to generate experience, nor does it need to be “broadcasted” globally. Although all the elements in the system are off (0), the system still forms a complex and specifies a MICS. Moreover, an element can contribute to experience as long as it affects the shape of the MICS, without the need to “broadcast” its activity globally to affect every other element. This is because information is not in the message that is broadcasted by an element, but it is the shape of the MICS that is specified by a complex.

Figure 19. Quantity and quality of experience of a “minimally conscious” photodiode.

(A) The minimally conscious photodiode DP consists of detector element D and predictor element P. D receives two external inputs and has a threshold ≥2. All connections have weight 1. (B) P serves as a memory for the previous state of D and its feed-back to D serves as a predictor of the next external input by effectively decreasing the threshold of D. (C) The MICS specified by the minimally conscious photodiode. D and P both specify a first order concept about the other element. (D) A minimally conscious thermistor or a minimally conscious blue detector with the same internal mechanisms as the minimally conscious photodiode generate the same MICS and therefore have the same minimal experience.

Figure 20. Feed-forward “zombie” systems do not generate consciousness.

(A) An unconscious photodiode DO without recurrent connections. The detector element D affects output element O, but has no cause within the system DO. O is caused by D, but has no effect on the photodiode DO. Therefore, the elements do not form a complex and generate no quale. (B) Even complicated systems cannot form a complex if they have a strictly feed-forward architecture. This can be understood in the following way: for any system background imposed by an observer, the system's input layer has no causes within the system and the output layer has no effects on it, regardless of the elements' (logic) functions. Consequently, the system cannot form a complex and it remains unconscious, just like the unconscious photodiode DO.

Figure 21. Functionally equivalent conscious and unconscious systems.

(A) A strongly integrated system gives rise to a complex in every network state. In the depicted state (yellow: 1, white: 0), elements form a complex with Φ^Max = 0.76 and 17 concepts. (B) Given many more elements and connections, it is possible to construct a feed-forward network implementing the same input-output function as the strongly integrated system in (A) for a certain number of time steps (here at least 4). This is done by unfolding the elements over time, keeping the memory of their past state in a feed-forward chain. The transition from the first layer to the second hidden layer in the feed-forward system is assumed to be faster than in the integrated system () to compensate for the additional layers (). Despite the functional equivalence, the feed-forward system is unconscious, a “zombie” without phenomenological experience, since its elements do not form a complex.

Figure 22. A complex can have ports-in and ports-out from and to the external environment, but its qualia are solipsistic: Self-generated, self-referential, and holistic.

(A) A recurrent segment/dot system consisting of 10 elements (8 linear threshold units, and 2 XOR logic gates) that are linked by excitatory and inhibitory connections (black +1, red −1). and C are the ports-in of the complex. They receive external inputs of strength 0, 1, or 2. Elements F and J are the ports-out of the complex. They output to the external elements O1 and O2. The current state of the system corresponds to a sustained input with value 2-2-0. From an extrinsic perspective, the different layers of the complex can be interpreted as feature detectors having increasingly invariant selectivities (e.g. D indicates “two contiguous left elements”, F “invariant segment”, and J “invariant dot”). (B) Since the segment/dot system is highly interconnected with specialized mechanisms, all first order concepts and many higher order concepts exist. (C) Both, elementary mechanisms that are “on” (1) and those that are “off” (0) constitute concepts. Note that the cause repertoire of is the mirror image of the cause repertoire of (highlighted in blue). (C,D,E) From the intrinsic perspective, the function of a mechanism is given by its cause-effect repertoire. The purview of a concept can only contain elements within the complex. The concepts that constitute the MICS generated by the complex are self-generated (specified exclusively by elements belonging to the complex); self-referential (specified exclusively over elements belonging to the complex); and holistic (their meaning is constructed in the context of the other concepts in the MICS).