EEG-based BCI Dataset of Semantic Concepts for Imagination and Perception Tasks

Abstract

Electroencephalography (EEG) is a widely-used neuroimaging technique in Brain Computer Interfaces (BCIs) due to its non-invasive nature, accessibility and high temporal resolution. A range of input representations has been explored for BCIs. The same semantic meaning can be conveyed in different representations, such as visual (orthographic and pictorial) and auditory (spoken words). These stimuli representations can be either imagined or perceived by the BCI user. In particular, there is a scarcity of existing open source EEG datasets for imagined visual content, and to our knowledge there are no open source EEG datasets for semantics captured through multiple sensory modalities for both perceived and imagined content. Here we present an open source multisensory imagination and perception dataset, with twelve participants, acquired with a 124 EEG channel system. The aim is for the dataset to be open for purposes such as BCI related decoding and for better understanding the neural mechanisms behind perception, imagination and across the sensory modalities when the semantic category is held constant.

Fig. 1

This figure shows an example of a pictorial trial. After a cue indicating whether the upcoming task is pictorial, orthographic or audio, five trials occur with a different stimulus used in each. Before the break, one block of each type of modality is cycled through, which takes around seven minutes. The duration of each break is chosen by the participant.

Fig. 2

Visualisation demonstrating that the three selected semantic words (penguin, guitar and flower) are semantically distant from each other. The distances, computed using Word2Vec, are plotted in 2D using t-SNE.

Fig. 3

Examples of the visual (a) pictorial and (b) orthographic stimuli used in the experiment. Pictorial stimuli ranged in complexity from simple to intermediate to naturalistic, while orthographic stimuli varied in colour and font.

Fig. 4

An example of a pictorial trial. After the cue, 5 trials occur with a different picture used in each. The picture is bounded in a white box, which reappears to frame the mental image for the imagination trial.

Fig. 5

Example of an orthographic trial. After the cue, 5 trials occur with a different orthographic representation used in each. The written word appears against a white background, which reappears in the imagination trial to ensure similar scaling between imagination and perception.

Fig. 6

Example of an auditory trial. After the cue, 5 trials occur with a different spoken word recording used in each. A white noise sound mask of 1000 ms is used to prevent residual stimulus audio representation leaking between the perception and imagination trials.

Fig. 7

The directory structure of the data according to BIDS format. Two versions of the EEG data are provided, raw and pre-processed versions.

Fig. 8

Displaying ERP for occipital regions including the electrodes: O1, O2, O1h, O2h, I1, Iz, I2, POO9, PO8, POO9b and POO10h. This is for participant 18, session 1.

Fig. 9

ITC for the six conditions averaged across participants. Specifically, ITC for (a) perceived audio, (b) perceived orthographic, (c) perceived pictorial, (d) imagined audio, (e) imagined orthographic and (f) imagined pictorial conditions. ITC is strongest in the perceived pictorial and orthographic conditions in the first 90 ms. ITC is weaker for imagination which is as expected due to the inter-trial variability in imagination generation and duration.

Fig. 10

The average power spectral density averaged over the 124 trials and the participants for each of the six conditions (a) perceived audio, (b) perceived orthographic, (c) perceived pictorial, (d) imagined audio, (e) imagined orthographic, (f) imagined pictorial.