Stable Decoding from a Speech BCI Enables Control for an Individual with ALS without Recalibration for 3 Months

Abstract

Brain-computer interfaces (BCIs) can be used to control assistive devices by patients with neurological disorders like amyotrophic lateral sclerosis (ALS) that limit speech and movement. For assistive control, it is desirable for BCI systems to be accurate and reliable, preferably with minimal setup time. In this study, a participant with severe dysarthria due to ALS operates computer applications with six intuitive speech commands via a chronic electrocorticographic (ECoG) implant over the ventral sensorimotor cortex. Speech commands are accurately detected and decoded (median accuracy: 90.59%) throughout a 3-month study period without model retraining or recalibration. Use of the BCI does not require exogenous timing cues, enabling the participant to issue self-paced commands at will. These results demonstrate that a chronically implanted ECoG-based speech BCI can reliably control assistive devices over long time periods with only initial model training and calibration, supporting the feasibility of unassisted home use.

Keywords: amyotrophic lateral sclerosis (ALS); brain-computer interfaces; neural decoding; speech brain-computer interface (BCI).

Figure 1

Schematics of the speech BCI for functional control. a) Neural signals were acquired from two 64‐channel ECoG arrays implanted over the motor and somatosensory areas responsible for upper extremity and speech functions. Only the inferior array was used in this study. b) A sample of high gamma energy (HGE, 70–170 Hz, z‐scored) for six channels. c) 1‐s rolling average of channel‐averaged HGE (updated every 10 ms). The peak of this signal was used to detect speech intent. Once the intended speech was detected, a decoding window consisting of HGE 2 s before and 0.5 s after the peak was sent to the classifier. d) The CNN model (InceptionTime^{[ 25 ]}) classified the window of HGE into commands that facilitated navigation of a communication board or control of external devices.

Figure 2

Stable performance of the BCI in online self‐paced experiments over 3 months. a) Online accuracy of the BCI system. Each dot represents one session. Average chance = 16.16% (n = 10000 simulations, dashed line). The blue line is the linear least squares regression line between accuracy and days after implant. b) Correct decoding results performed by the BCI per minute. Each dot represents one session. The blue line is the linear least squares regression line between correct decodes per minute and days after implant. c) Number of false detections (blue dot) and missed detections (purple triangle) per minute. Each symbol represents one experiment session. d) Time between speech offset and when the decoding result was registered by the BCI system for every successful decode per day. For all boxplots, the center line represents the median, top and bottom edges of the box represent quantiles. Data outside of 1.5 times of interquartile range were shown as outlier data points, and the maximum and minimum of non‐outliers were shown as whiskers.

Figure 3

Stability of the event‐related high gamma activities acquired from the ECoG arrays. a) Anatomical location of the ECoG array used in this study. Example channels in (b) are highlighted. b) Examples of event‐related HGE in both training and real‐time usage phases for two different commands. A vertical dotted line at 0 s indicates speech onset. The shaded area represents 95% CI. CB: Communication Board (real‐time usage). WP: Word Production (training data). c) Correlation between the real‐time usage trials and average training data per channel. For each real‐time usage trial, the Pearson's correlation coefficient between its HGE and the average HGE of the corresponding command in training data collection phase was calculated. Each dot represents the average (weighted against the frequency of the command) of the correlation coefficients per usage day per channel. The blue line represents the linear least squares regression line between channel‐averaged correlation and days after implant. d) Rate of change for correlation. Each dot represents one channel. Filled dots represent statistically significant linear relationships between correlation values and days after implant (p <0.05, Wald test with t‐distribution). Unfilled dots indicate that a relationship could not be established (p > = 0.05). e) Channel‐average of logarithmic HGE (unnormalized) for each command during online usage. Lines represent the linear least squares regression lines between HGE and days after implant for each command. f) Same as (d), but for HGE.

Figure 4

Electrode contribution during the study period. a) MRI reconstruction of the participant's brain, overlaid on top of which are the ECoG grids implanted as part of the clinical trial. Electrodes used in this study are colored in red (motor) and blue (sensory). The grey electrodes were not used in this study. b) Simulated online accuracy when the decoding model is trained with both motor and sensory electrodes, only motor electrodes, only sensory electrodes, and only the most salient electrode. Chance = 16.67% (shown as dashed line). Each box corresponds to the accuracy for n = 33 testing days (^****p < 0.0001, Mann–Whitney‐Wilcoxon test two‐sided with Bonferroni correction). c) Relative contribution of each of the electrodes to the decoding results for each real‐time usage month.

Figure 5

Performance in functional control and during silent (mimed) speech. a) Online accuracy of the BCI system during functional control. Each data point represents one session. Chance = 16.67% (dashed line). The blue line is the linear least squares regression line between accuracy and days after implant. b) Correct decoding results performed by the BCI per minute during functional control. Each dot represents one session. The blue line is the linear least squares regression line between correct decodes per minute and days after implant. c) Number of false detections (blue dots) and missed detections (purple triangles) per minute during function control. Each symbol represents one session. d) Online accuracy of silent speech decoder. Each dot represents one day. Average chance = 16.73% (dashed line, n = 10000 simulations). The purple line represents the linear least squares regression between accuracy and days after implant. e) Correct decoding results performed by the BCI per minute using the silent speech decoder. Each dot represents one day. The purple line is the linear least squares regression line between correct decodes per minute and days after implant.