Using data from cue presentations results in grossly overestimating semantic BCI performance

Abstract

Neuroimaging studies have reported the possibility of semantic neural decoding to identify specific semantic concepts from neural activity. This offers promise for brain-computer interfaces (BCIs) for communication. However, translating these findings into a BCI paradigm has proven challenging. Existing EEG-based semantic decoding studies often rely on neural activity recorded when a cue is present, raising concerns about decoding reliability. To address this, we investigate the effects of cue presentation on EEG-based semantic decoding. In an experiment with a clear separation between cue presentation and mental task periods, we attempt to differentiate between semantic categories of animals and tools in four mental tasks. By using state-of-the-art decoding analyses, we demonstrate significant mean classification accuracies up to 71.3% during cue presentation but not during mental tasks, even with adapted analyses from previous studies. These findings highlight a potential issue when using neural activity recorded during cue presentation periods for semantic decoding. Additionally, our results show that semantic decoding without external cues may be more challenging than current state-of-the-art research suggests. By bringing attention to these issues, we aim to stimulate discussion and drive advancements in the field toward more effective semantic BCI applications.

Keywords: Brain-computer interface (BCI); Cue presentation; Electroencephalography (EEG); Functional magnetic resonance imaging (fMRI); Machine learning; Mental imagery; Semantic decoding.

Fig. 1

Illustration of one concept trial in Datasets 1 and 2. The order of mental tasks is randomized across blocks in Dataset 1.

Fig. 2

Mean classification accuracies when classifiers can utilize information from all channels and all time points of corresponding periods. Horizontal lines represent significant borderlines for $p=0.05$ (solid), $p=0.01$ (dashed), and $p=0.001$ (dotted).

Fig. 3

Single-channel mean classification accuracies above the significance borderline ($p=0.05$) for the image presentation when classifiers can utilize information from all time points of the image presentation period. White represents non-significant classification accuracies.

Fig. 4

Mean classification accuracies during the image presentation period when the classifier can use information from all channels. Classification accuracies are shown with mean and 95% confidence interval. Horizontal lines represent significant borderlines for $p=0.05$ (solid), $p=0.01$ (dashed), and $p=0.001$ (dotted).

Fig. 5

Mean classification accuracies during the image presentation period for each channel using the SVM (CV). Times represent the start and end of each temporal window after the image onset. Scalp maps indicate performance above the significance borderline ($p=0.05$). White represents non-significant classification accuracies.

Fig. 6

Mean classification accuracies from adapted analysis in which classifiers can utilize information from all channels and the first 600 ms after the period onset. Horizontal lines represent significant borderlines for $p=0.05$ (solid), $p=0.01$ (dashed), and $p=0.001$ (dotted).

Fig. 7

Mean classification accuracies from adapted analysis in which the SVM (CV) can utilize a certain number of CSP components. Horizontal lines represent significant borderlines for $p=0.05$ (solid), $p=0.01$ (dashed), and $p=0.001$ (dotted).