How do inner screens enable imaginative experience? Applying the free-energy principle directly to the study of conscious experience

Abstract

This paper examines the constraints that the free-energy principle (FEP) places on possible model of consciousness, particularly models of attentional control and imaginative experiences, including episodic memory and planning. We first rehearse the classical and quantum formulations of the FEP, focusing on their application to multi-component systems, in which only some components interact directly with the external environment. In particular, we discuss the role of internal boundaries that have the structure of Markov blankets, and hence function as classical information channels between components. We then show how this formal structure supports models of attentional control and imaginative experience, with a focus on (i) how imaginative experience can employ the spatio-temporal and object-recognition reference frames employed in ordinary, non-imaginative experience and (ii) how imaginative experience can be internally generated but still surprising. We conclude by discussing the implementation, phenomenology, and phylogeny of imaginative experience, and the implications of the large state and trait variability of imaginative experience in humans.

Keywords: Aphantasia; Cognitive architecture; Depression; Inner speech; Introspection; Metacognition; Planning; Visual imagery.

Figure 1.

Illustration of a Markov blanket (MB). Internal and external (i.e. environmental) states (denoted η and µ, respectively) interact via sensory and active blanket states (denoted s and a, respectively, with the total blanket state denoted b = (s,a)); the latter induce a conditional independence between internal and external states. In virtue of this conditional independence, one can associate internal states with the parameters or sufficient statistics of a conditional (a.k.a., variational) density over external states, given blanket states. In turn, one can then interpret internal (and active) dynamics as a gradient flow on a variational free energy that plays the role of a marginal likelihood or model evidence in statistical inference. Equations represent time derivatives (the notation ; is the gradient operator) of the state variables as functions of each other. The vertical solid arrow emphasizes that the time evolution of both sensory and active states depends on the total blanket state b.

Figure 2.

“Attaching” a QRF represented as a hierarchy of operators to an intersystem boundary depicted as an ancillary array of qubits q_i. Operators , k = S or E, are single-bit components of the interaction Hamiltonian H_SE. The node is both the limit and the colimit of maps from and to the nodes ; only leftward-going (cocone implementing) arrows are shown for simplicity. Adapted from Fields and Glazebrook (2020), CC-BY license.

Figure 3.

A minimum architecture for an agent with multiple QRFs (light blue rectangles) and attentional control via TFE resource distribution implemented by an executive/metacognitive system (orange rectangle). Both the agent S as a whole and the executive/metacognitive component have boundaries/MBs; the MB of the executive/metacognitive component is an “inner screen.” The thermodynamic sector of each boundary/MB, as well as the TFE distribution system are shown by grey hatching; the informative sectors of boundaries are dull blue. Red vertical arrow indicates that TFE inputs to some QRFs, e.g. homeostatic interoception, can over-ride attenuation by the executive/metacognitive system. Equating what is experienced with what is encoded on the informative sector of the boundary/MB, the overall agent S experiences sensations from and actions on its environment E, while its executive/metacognitive component experiences inputs from and outputs to the QRFs labeled 1, 2, . Neither S nor its executive/metacognitive component have imaginative experiences, i.e. experiences of something other than their interactions with their own environments.

Figure 4.

Adding an inner screen and a control layer to the architecture shown in Fig. 3 enables imaginative experiences that employ the same QRFs as non-imaginative experience. The agent S can, with this architecture, choose to perceive and act on the external environment E or on an “imagined environment” encoded only on . Horizontal red arrows indicate that some external inputs can over-ride being attenuated by the executive/metacognitive system; this over-ride capability can be expected to be both state- and trait-variable.