Speech imagery brain-computer interfaces: a systematic literature review

Abstract

Objective:Speech Imagery (SI) refers to the mental experience of hearing speech and may be the core of verbal thinking for people who undergo internal monologues. It belongs to the set of possible mental imagery states that produce kinesthetic experiences whose sensations are similar to their non-imagery counterparts. SI underpins language processes and may have similar building blocks to overt speech without the final articulatory outcome. The kinesthetic experience of SI has been proposed to be a projection of the expected articulatory outcome in a top-down processing manner. As SI seems to be a core human cognitive task it has been proposed as a paradigm for Brain-Computer Interfaces (BCI). One important aspect of BCI designs is usability, and SI may present an intuitive paradigm, which has brought the attention of researchers to attempt to decode SI from brain signals. In this paper we review the important aspects of SI-BCI decoding pipelines.Approach. We conducted this review according to the Preferred Reporting Items for Systematic reviews and Meta-Analysis guidelines. Specifically, we filtered peer-reviewed reports via a search of Google Scholar and PubMed. We selected a total of 104 reports that attempted to decode SI from neural activity.Main results. Our review reveals a growing interest in SI decoding in the last 20 years, and shows how different neuroimaging modalities have been employed to record SI in distinct ways to instruct participants to perform this task. We discuss the signal processing methods used along with feature extraction techniques and found a high preference for Deep Learning models. We have summarized and compared the decoding attempts by quantifying the efficacy of decoding by measuring Information Transfer Rates. Notably, fewer than 6% of studies reported real-time decoding, with the vast majority focused on offline analyses. This suggests existing challenges of this paradigm, as the variety of approaches and outcomes prevents a clear identification of the field's current state-of-the-art. We offer a discussion of future research directions.SignificanceSI is an attractive BCI paradigm. This review outlines the increasing interest in SI, the methodological trends, the efficacy of different approaches, and the current progress toward real-time decoding systems.

Keywords: ECoG; EEG; brain–computer interfaces; fNIRS; inner speech; speech imagery; systematic literature review.