Don't put words in my mouth: speech perception can falsely activate a brain-computer interface

Abstract

Background: Recent studies have demonstrated that speech can be decoded from brain activity which in turn can be used for brain-computer interface (BCI)-based communication. It is however also known that the area often used as a signal source for speech decoding BCIs, the sensorimotor cortex (SMC), is also engaged when people perceive speech, thus making speech perception a potential source of false positive activation of the BCI. The current study investigated if and how speech perception may interfere with reliable speech BCI control.

Methods: We recorded high-density electrocorticography (HD-ECoG) data from five subjects while they performed a speech perception and a speech production task. We first evaluated whether speech perception and production activated the SMC. Second, we trained a support-vector machine (SVM) on the speech production data (including rest). To test the occurrence of false positives, this decoder was then tested on speech perception data where every perception segment that was classified as a produced syllable rather than rest was considered a false positive. Finally, we investigated whether perceived speech could be distinguished from produced speech and rest.

Results: Our results show that both the perception and production of speech activate the SMC. In addition, we found that decoders that are highly reliable at detecting self-produced syllables from brain signals may generate false positive BCI activations during the perception of speech and that it is possible to distinguish perceived speech from produced speech and rest, with high accuracy.

Conclusions: We conclude that speech perception can interfere with reliable BCI control, and that efforts to limit the occurrence of false positives during daily-life BCI use should be implemented in BCI design to increase the likelihood of successful adoptation by end users.

Keywords: ALS; Brain-Computer interface; Communication; Locked-in syndrome; Neuroprosthesis; Reliability; Speech decoding.

Fig. 1

Grid placement. (A-E) Grid placement on individual surface reconstructions for participants 1–5. Central sulcus is indicated with a black line. Grey electrodes are excluded because they showed poor signal quality or were outside of the SMC. (F) Electrode placement (coordinates converted to MNI space) of all included electrodes of all participants, projected to an MNI cortex reconstruction. Central sulcus and Sylvian fissure are indicated with black lines

Fig. 2

Significantly responding electrodes to the tasks for all subjects. Electrodes are visualized on an MNI brain. Central sulcus and sylvian fissure are indicated with black lines. Different participants are indicated in different colors. Speech perception electrodes that are responsive during at least one version of production are indicated with a white rim (left panel), as are production electrodes that are also responsive during speech perception (middle and right panel)

Fig. 3

Classification scores and False Positive rates. (A, C) Classification scores on produced speech epochs for each participant using an overt-based (dark green) and mimed-based (light green) decoder, before (A) and after (C) rest normalization. Dashed lines indicate the chance level, which is the mean classification accuracy of the distribution generated using random labels. (B, D) Percentages of false positively classified epochs for the decoder based on the overt and mimed speech separately, before (B) and after (D) rest normalization. The percentage scores of false positive classified epochs are visualized for the perception epochs (left), the perception-rest epochs (middle), and the production-rest epochs (right). Subjects are indicated with different markers. Red bars indicate significant differences, with * indicating p <.05

Fig. 4

Decoding performance in distinguishing perception from production and rest. Classification scores on perceived, produced, and rest epochs using an overt-based (dark green) and mimed-based (light green) decoder, Dashed lines indicate the chance level, which is the mean classification accuracy of the distribution generated using random labels