Offline analysis of context contribution to ERP-based typing BCI performance

Abstract

Objective: We aim to increase the symbol rate of electroencephalography (EEG) based brain-computer interface (BCI) typing systems by utilizing context information.

Approach: Event related potentials (ERP) corresponding to a stimulus in EEG can be used to detect the intended target of a person for BCI. This paradigm is widely utilized to build letter-by-letter BCI typing systems. Nevertheless currently available BCI typing systems still require improvement due to low typing speeds. This is mainly due to the reliance on multiple repetitions before making a decision to achieve higher typing accuracy. Another possible approach to increase the speed of typing while not significantly reducing the accuracy of typing is to use additional context information. In this paper, we study the effect of using a language model (LM) as additional evidence for intent detection. Bayesian fusion of an n-gram symbol model with EEG features is proposed, and a specifically regularized discriminant analysis ERP discriminant is used to obtain EEG-based features. The target detection accuracies are rigorously evaluated for varying LM orders, as well as the number of ERP-inducing repetitions.

Main results: The results demonstrate that the LMs contribute significantly to letter classification accuracy. For instance, we find that a single-trial ERP detection supported by a 4-gram LM may achieve the same performance as using 3-trial ERP classification for the non-initial letters of words.

Significance: Overall, the fusion of evidence from EEG and LMs yields a significant opportunity to increase the symbol rate of a BCI typing system.

Figure 1

Sample visual presentation screens; matrix presentation (on the left) and rapid serial visual presentation (on the right)

Figure 2

An example of the change in AUC while searching shrinkage (λ) and regularization (γ). Highest AUC is obtained for λ = 0.6 and γ = 0.1.

Figure 3

An example of an ROC curve corresponding to single sequence and 0-gram language model.

Figure 4

The average correct letter selection probability vs inverse of the number of repetitions for various language model orders and letter locations in the word.

Figure 5

The average expected value of typing duration vs inverse of the number of repetitions for various language model orders and letter locations in the word. In the graphs 100 seconds mark is used jointly with the failure case. If the subject has a probability of getting stuck it is considered as a failure.

Figure 6

HS1 number of sequences used to type each symbol. If there are multiple epochs needed to correctly type a symbol, unintended decisions and backspaces to correct it are represented with different colors. Typing duration was 17.8 and 23.2 seconds/symbol, respectively.

Figure 7

HS2 number of sequences used to type each symbol. If there are multiple epochs needed to correctly type a symbol, unintended decisions and backspaces to correct it are represented with different colors. Typing duration was 17.1 and 23.8 seconds/symbol, respectively.