Classification of intended phoneme production from chronic intracortical microelectrode recordings in speech-motor cortex

Abstract

We conducted a neurophysiological study of attempted speech production in a paralyzed human volunteer using chronic microelectrode recordings. The volunteer suffers from locked-in syndrome leaving him in a state of near-total paralysis, though he maintains good cognition and sensation. In this study, we investigated the feasibility of supervised classification techniques for prediction of intended phoneme production in the absence of any overt movements including speech. Such classification or decoding ability has the potential to greatly improve the quality-of-life of many people who are otherwise unable to speak by providing a direct communicative link to the general community. We examined the performance of three classifiers on a multi-class discrimination problem in which the items were 38 American English phonemes including monophthong and diphthong vowels and consonants. The three classifiers differed in performance, but averaged between 16 and 21% overall accuracy (chance-level is 1/38 or 2.6%). Further, the distribution of phonemes classified statistically above chance was non-uniform though 20 of 38 phonemes were classified with statistical significance for all three classifiers. These preliminary results suggest supervised classification techniques are capable of performing large scale multi-class discrimination for attempted speech production and may provide the basis for future communication prostheses.

Keywords: chronic recording; locked-in syndrome; motor cortex; neurotrophic electrode; speech prosthesis.

Figure 1

The power induction coil is on the right side, with the two FM receiving coils on the left. All coils are held in position with EC2 electrode paste.

Figure 2

Illustration of extracellular recording preprocessing steps. Left: (top) 1000 ms sample of the wideband (1–9000 Hz) signal. The shaded box region is 100 ms long and is shown zoomed-in and filtered between 300 and 6000 Hz below; (bottom) 100 ms zoomed-in band-pass filtered signal with marked clustered spike events. Circles represent cluster 301, diamonds cluster 302, and squares cluster 305. Cluster identifiers correspond to the cluster label (tens and ones digits) per analysis stream (hundreds digit). In this example, clusters 1, 2, and 5 are shown from analysis stream 3. Right: Average clustered spike event waveforms (black) with individual spike event waveform traces in gray for three representative clusters.

Figure 3

Timing of training paradigm for phoneme detection trials is as follows: [1] “listen” instruction (t = 0 s), [2] “listen” tone (acoustic, t = 0. 75 s), [3] acoustic stimulus presentation (t = 1.5 s), [4] “speak” instruction (t = 2.25 s), [5] “speak” tone (t = 3.0 s), and [6] subject is instructed to produce the presented phoneme. The response period is 10 s long.

Figure 4

The classification accuracy for each phoneme using the LDA, SVM, and FDA classifiers is shown with the theoretical chance performance level (dashed line, ∼1/38). Phonemes with classification rates statistically above chance (binomial test, Bonferroni-adjusted p < 0.05) are marked with an asterisk (*). Twenty of 38 phonemes were classified above chance for all methods, and 24 phonemes for at least two of the three classification methods and 26 phonemes for any.

Figure 5

Confusion matrices (Eexpected Phoneme on y-axis, Classified Phoneme on x-axis) are displayed summarizing the cross-validated performance of each classification technique for all phonemes (N = 38). Each matrix represents one of the classification methods: LDA, SVM, and FDA. Every other phoneme is labeled for readability. The confusion matrix color scale is normalized across methods and increases from white to black as a function of increasing classification rate.