Brain-wave recognition of sentences

Abstract

Electrical and magnetic brain waves of two subjects were recorded for the purpose of recognizing which one of 12 sentences or seven words auditorily presented was processed. The analysis consisted of averaging over trials to create prototypes and test samples, to each of which a Fourier transform was applied, followed by filtering and an inverse transformation to the time domain. The filters used were optimal predictive filters, selected for each subject. A still further improvement was obtained by taking differences between recordings of two electrodes to obtain bipolar pairs that then were used for the same analysis. Recognition rates, based on a least-squares criterion, varied, but the best were above 90%. The first words of prototypes of sentences also were cut and pasted to test, at least partially, the invariance of a word's brain wave in different sentence contexts. The best result was above 80% correct recognition. Test samples made up only of individual trials also were analyzed. The best result was 134 correct of 288 (47%), which is promising, given that the expected recognition number by chance is just 24 (or 8.3%). The work reported in this paper extends our earlier work on brain-wave recognition of words only. The recognition rates reported here further strengthen the case that recordings of electric brain waves of words or sentences, together with extensive mathematical and statistical analysis, can be the basis of new developments in our understanding of brain processing of language.

Figure 1

Shaded contour map of recognition-rate surface for the 10–20 system of EEG sensors for subject S8. The physical surface of the scalp is represented as a plane circle as is standard in representations of the 10–20 system. The recognition rate for each sensor is shown, next to it, as the number of test samples of sentences correctly recognized of a total of 60. The predictions are for the best parameters L = 3 Hz, W = 7 Hz, s = 88 ms, and e = 2,455 ms.

Figure 2

Contour map of recognition-rate surface for filter parameters L and W for subject S8. The x coordinate is the low frequency (L) in Hz and the y coordinate is the width (W) in Hz of a filter. The number plotted as a point on the map is the number of test samples of sentences correctly recognized of a total of 60 by the Butterworth filter with the coordinates of the point. The predictions are for the best temporal parameters s = 88 ms, e = 2,455 ms, and best sensor C3.

Figure 3

Contour map of recognition-rate surface for start (s) and ending (e) parameters for subject S8. The x coordinate is the start point (s), measured in ms, of the interval of observations, used for prediction. The y coordinate is the ending point (e), measured in ms, of the same interval. The recognition-rate numbers have the same meaning as in Figs. 1 and 2. The predictions are for the best filter parameters L = 3 Hz and W = 7 Hz, and best sensor C3.

Figure 4

Speech spectrum and corresponding brain wave of sentence Bill sees Susan. The x axis is measured in ms after onset of spoken-sentence stimulus. The speech and brain waves have different amplitudes, so no common scale is shown on the y axis.

Figure 5

Brain waves of S8 (dotted curved line) and S9 (solid curved line) for spoken word John. In the case of S8, the spoken word occurred as the first word of a sentence. The x axis is measured in ms after onset of the auditory stimulus.