Brain-Computer Interfaces for Speech Communication

Abstract

This paper briefly reviews current silent speech methodologies for normal and disabled individuals. Current techniques utilizing electromyographic (EMG) recordings of vocal tract movements are useful for physically healthy individuals but fail for tetraplegic individuals who do not have accurate voluntary control over the speech articulators. Alternative methods utilizing EMG from other body parts (e.g., hand, arm, or facial muscles) or electroencephalography (EEG) can provide capable silent communication to severely paralyzed users, though current interfaces are extremely slow relative to normal conversation rates and require constant attention to a computer screen that provides visual feedback and/or cueing. We present a novel approach to the problem of silent speech via an intracortical microelectrode brain computer interface (BCI) to predict intended speech information directly from the activity of neurons involved in speech production. The predicted speech is synthesized and acoustically fed back to the user with a delay under 50 ms. We demonstrate that the Neurotrophic Electrode used in the BCI is capable of providing useful neural recordings for over 4 years, a necessary property for BCIs that need to remain viable over the lifespan of the user. Other design considerations include neural decoding techniques based on previous research involving BCIs for computer cursor or robotic arm control via prediction of intended movement kinematics from motor cortical signals in monkeys and humans. Initial results from a study of continuous speech production with instantaneous acoustic feedback show the BCI user was able to improve his control over an artificial speech synthesizer both within and across recording sessions. The success of this initial trial validates the potential of the intracortical microelectrode-based approach for providing a speech prosthesis that can allow much more rapid communication rates.

Figure 1

Examples of the four intracortical electrodes discussed in this paper for use in chronic BCI applications. (A) Microwire array; Nicolelis et al. (2003). Chronic, multisite, multielectrode recordings in macaque monkeys. Proceedings of the National Academy of Sciences of the United States of America, 100(19), 11041–11046. Copyright (2003) National Academy of Sciences, U.S.A., (B) Michigan microelectrode array (Wise et al., 2004), reproduced with permission (© 2004 IEEE), (C) Utah microelectrode array; Reprinted from Journal of Neuroscience Methods, Vol. 82 (1), Rousche and Normann, Chronic recording capability of the Utah Intracortical Electrode Array in Cat Sensory Cortex, 1–15, Copyright (1998) with permission from Elsevier, (D) Neurotrophic Electrode; Reprinted from Journal of Neuroscience Methods, Vol. 174 (2), Bartels et al., Neurotrophic Electrode: Method of assembly and implantation into human motor speech cortex, 168–176, Copyright (2008) with permission from Elsevier.

Figure 2

Schematic of the continuous neural decoder for speech synthesis. Neural signals are obtained via the Neurotrophic Electrode and acquired utilizing the Cheetah acquisition system (Neuralynx, Inc., Bozeman, MT). Formant frequencies or articulatory trajectories are decoded from neural firing rates.

Figure 3

Two dimensional (F1/F2) tuning preferences for sample units determined from offline formant analysis in Guenther et al. (in press). Tuning preference determined by correlation between unit firing rates and formant frequency trajectories. Mean tuning preference indicated by the black curve with 95% confidence intervals in gray. (a,b) tuning curves for units with primarily F2 preference. (c,d) tuning curves for units with mixed F1 – F2 preference. The direction and magnitude of each tuning curve indicates the preference for formant frequencies relative to the center vowel /AH/.

Figure 4

The two dimensional histogram of predicted formant trajectories found during the closed-loop BCI study. Trajectories for /AH AA/ trials are shown in red, /AH UW/ in green and /AH IY/ in blue. Formant regions used for vowel classification and attractors for steady vowel production shown in gray. (a) All trials from 25 closed-loop recording sessions. (b) Correct trials only from all recording sessions.