Language model applications to spelling with Brain-Computer Interfaces

Abstract

Within the Ambient Assisted Living (AAL) community, Brain-Computer Interfaces (BCIs) have raised great hopes as they provide alternative communication means for persons with disabilities bypassing the need for speech and other motor activities. Although significant advancements have been realized in the last decade, applications of language models (e.g., word prediction, completion) have only recently started to appear in BCI systems. The main goal of this article is to review the language model applications that supplement non-invasive BCI-based communication systems by discussing their potential and limitations, and to discern future trends. First, a brief overview of the most prominent BCI spelling systems is given, followed by an in-depth discussion of the language models applied to them. These language models are classified according to their functionality in the context of BCI-based spelling: the static/dynamic nature of the user interface, the use of error correction and predictive spelling, and the potential to improve their classification performance by using language models. To conclude, the review offers an overview of the advantages and challenges when implementing language models in BCI-based communication systems when implemented in conjunction with other AAL technologies.

Figure 1.

Brain-Computer Interface scheme.

Figure 2.

Example of typing matrix in P300-speller. Rows and columns are intensified in random order. The intensification of the second column (Left panel) and the third row (Right panel) are shown. One round consists of the intensification of each one of the six columns and six rows.

Figure 3.

Three consecutive stages to select symbol “w” in a SSVEP speller.

Figure 4.

Two different types of imaginary movements allow the user to control the rotation and extension of the gray arrow in the Hex-o-Spell interface. In this example the last letter in the word “BELARUS” is selected. Adapted from [99].

Figure 5.

Layout of the Bremen SSVEP-based BCI-speller. Adapted from [101].

Figure 6.

Conventional (a) and modified (b) P300-speller interfaces used in the study of Ahi and colleagues [103]. Adapted from [103].

Figure 7.

Dynamic (a) and static (b) interfaces for a system, that considers previously typed text “what a w” for spelling the next letter. Adapted from [98].

Figure 8.

Five successive stages when entering “Hello” with the Dasher interface.

Figure 9.

Two different layouts designed for predictive spelling. (a) The predicted words are displayed on the left side of the screen over an “extra” window in the interface, thus requiring keeping them in the user's memory, which could increase the cognitive workload. Adapted from [114]. (b) The solution proposed to alleviate the cognitive workload by integrating the suggested words into the interface as selectable stimuli. Adapted from [115].