Synchronised neural signature of creative mental imagery in reality and augmented reality

Abstract

Creativity, transforming imaginative thinking into reality, is a mental imagery simulation in essence. It can be incorporeal, concerns sophisticated and/or substantial thinking, and involves objects. In the present study, a mental imagery task consisting of creating a scene using familiar (FA) or abstract (AB) physical or virtual objects in real (RMI) and augmented reality (VMI) environments, and an execution task involving effectively creating a scene in augmented reality (VE), were utilised. The beta and gamma neural oscillations of healthy participants were recorded via a 32 channel wireless 10/20 international EGG system. In real and augmented environments and for both the mental imagery and execution tasks, the participants displayed a similar cortico-cortical neural signature essentially based on synchronous vs asynchronous beta and gamma oscillatory activities between anterior (i.e. frontal) and posterior (i.e. parietal, occipito-parietal and occipito-temporal) areas bilaterally. The findings revealed a transient synchronised neural architecture that appears to be consistent with the hypothesis according to which, creativity, because of its inherent complexity, cannot be confined to a single brain area but engages various interconnected networks.

Keywords: Augmented reality; Complexity; Creativity; Mental imagery; Real environment; Synchronisation.

Figure 1

A: Familiar objets (FA) were selected from a pre-existing application (Hololens) but only the six most frequently occurring objects in the English corpus as by Google Ngram Viewer were considered. There were all namable: a key, a star, a painting, a frame, a chair and a plant. B: Abstract objects (AB) were created using software Unity (version 2018), there were similar to the FA in terms of shape, colour and size. All ABs objects were not namable. The objects (FA and AB) were presented either physically, on a A4 paper or as virtual incorporated in the 3D augmented reality environment via HolonLens 1st generation.

Figure 2

The experiment phase comprises three conditions: RMI (Real Mental Imagery), VMI (Virtual Mental Imagery) and VE (Virtual Execution). In the RMI participants were equipped with the mobita EEG, in the VMI and VE conditions participants were outfitted with EEG and the AR (Hololens) device. In all experimental conditions participants standing in the experimental room (i.e. on a white "X"). During the RMI, the participants were instructed to imagine creating a scene in their mind by pinching and grasping all six FA or AB objects one-by-one independently and putting them together. During VMI condition, the participants were immersed in AR via the HolonLens. As previously, participants were asked to imagine pinching and grasping each FA and AB virtual object and create a scene within the augmented reality environment. In the Virtual Execution (VE) condition, the participants were instructed to use the FA and AB incorporated virtual objects to create a scene in the 3D augmented environment. In each condition, the participants were told to remain in the same position (i.e. indicated by a white X), not to move their head and were not allowed to resize, rotate, or break apart the object and use all the objects. In each condition (RMI, VMI and VE), participants disposed of 2 minutes and 30 seconds to create the scene with FA objects, and another 2 minutes 30 seconds to create the scene with AB objects. The inter-condition duration time was 2 minutes approximately. At the end of each time period and for both FA and AB objects within each experimental condition, the participants were asked to give a title to their scenes. Continuous EEG recording was obtained for the creation of the scenes with FA and AB separately, i.e. 2 minutes and 30 seconds for FA and 2 minutes and 30 seconds for AB in each experimental condition: RMI, VMI, and VE.

Figure 3

Neuroanatomical connection network for beta oscillations (12.5 to 30 Hz). Top: The network includes the frontal, parietal cortices and the occipito-temporal and occipito-parietal pathways of the occipital cortex in RMI, VMI and VE experimental conditions. Bottom: Functional connectivity network represents cross-correlations of the regional electrical signal, as estimated from simulated mathematical model dynamics. The colour bar gives the Pierson correlations values. Red colours represent positive correlations, while blue colours represent negative correlations. F: frontal, P: Parietal, OP: Occipito-parietal, OT: Occipito-temporal. In summary, within each experimental condition (RMI, VMI and VE), positive correlations of beta oscillations were observed between bilateral posterior areas, and negative correlations between bilateral anterior and posterior areas.

Figure 4

Neuroanatomical connection network for gamma oscillations (55 to 80 Hz). Top: The network includes the frontal, parietal cortices and the occipito-temporal and occipito-parietal pathways of the occipital cortex in RMI, VMI and VE experimental conditions. Bottom: Functional connectivity network represents cross-correlations of the regional electrical signal, as estimated from simulated mathematical model dynamics. The colour bar gives the Pierson correlations values. Red colours represent positive correlations, while blue colours represent negative correlations. F: frontal, P: Parietal, OP: Occipito-parietal, OT: Occipito-temporal. In summary, when gamma oscillations were analysed positive correlations characterised the bilateral posterior areas in VMI and VE conditions, and negative correlations the bilateral posterior and anterior areas in all experimental conditions (RMI, VMI and VE).

Figure 5

Loadings derives from PCA (Principal Component Analysis) for beta oscillations. The PCA revealed three factors accounting for 63.94% of the total variance and all four areas of interest, i.e. frontal, parietal, occipito-parietal and occipito-temporal were included in the model. There was more beta oscillations neural activation in parietal, occipito-temporal and ocipito-parietal areas, and less activation in frontal areas during the creative task in RMI (Real Mental Imagery), VMI (Virtual Mental Imagery) and VE (Virtual Execution) conditions.

Figure 6

Loadings derives from PCA (Principal Component Analysis) for gamma oscillations. Three factors which explained 71.35% of the total variance were reported with the all four regions of interest included in the model. There was more gamma oscillations neural activation in occipito-temporal and occipito-parietal areas during the creative task in all experimental conditions (RMI-Real Mental Imagery, VMI-Virtual Mental Imagery and VE-Virtual Execution). Similarly, more neural activity in parietal and frontal areas was observed in VMI (Virtual Mental Imagery) and VE (Virtual Execution) condition respectively. However, there was less gamma oscillations in a frontal and parietal areas in RMI (Real Mental Imagery) condition, in frontal areas in VMI (Virtual Mental Imagery) condition and in parietal areas in VE (Virtual Execution) condition.