Rapid Communication with a "P300" Matrix Speller Using Electrocorticographic Signals (ECoG)

Abstract

A brain-computer interface (BCI) can provide a non-muscular communication channel to severely disabled people. One particular realization of a BCI is the P300 matrix speller that was originally described by Farwell and Donchin (1988). This speller uses event-related potentials (ERPs) that include the P300 ERP. All previous online studies of the P300 matrix speller used scalp-recorded electroencephalography (EEG) and were limited in their communication performance to only a few characters per minute. In our study, we investigated the feasibility of using electrocorticographic (ECoG) signals for online operation of the matrix speller, and determined associated spelling rates. We used the matrix speller that is implemented in the BCI2000 system. This speller used ECoG signals that were recorded from frontal, parietal, and occipital areas in one subject. This subject spelled a total of 444 characters in online experiments. The results showed that the subject sustained a rate of 17 characters/min (i.e., 69 bits/min), and achieved a peak rate of 22 characters/min (i.e., 113 bits/min). Detailed analysis of the results suggests that ERPs over visual areas (i.e., visual evoked potentials) contribute significantly to the performance of the matrix speller BCI system. Our results also point to potential reasons for the apparent advantages in spelling performance of ECoG compared to EEG. Thus, with additional verification in more subjects, these results may further extend the communication options for people with serious neuromuscular disabilities.

Keywords: P300; brain–computer interface; electrocorticography; event-related potential; speller.

Figure 1

Implant. The subject had 96 subdural electrodes (two grids and two strips in different configurations) implanted over left frontal, parietal, temporal, and occipital regions. (A) Photograph of the craniotomy and the implanted grids in this subject. (B) Lateral X-ray of the subject, showing an 8 × 8 grid over frontal/parietal cortex, a 23-contact grid over temporal cortex, and several strips.

Figure 2

Experimental setup. The subject sat 60 cm in front of a flat-screen monitor that presented a centered 6 × 6 matrix containing alphanumeric characters as well as space (Sp) and backspace (Bs). The rows and columns in the matrix flashed rapidly and pseudo-randomly. The subject's task was to pay attention to the intended character. The computer determined the intended character from the subject's ECoG responses.

Figure 3

Event-related potentials (ERPs). The figure above shows averaged event-related responses to target (red) and non-target (blue) flashes at each of the 96 recorded locations.

Figure 4

Optimizing accuracy and information transfer rate. The figure on the left shows the relationship between the flash duration and letter classification accuracy with a single-flash sequence. The figure on the right shows the relationship between the number of flash sequences and classification accuracy using a flash duration of 3/64 s (i.e., 47 ms). The subject reached a maximum of 98% classification accuracy at three flash sequences, and a maximum of 60.5 bits/min at 92.2% accuracy (i.e., a selection every 4.5 s) at two flash sequences.

Figure 5

Qualitative results. The figure at the top shows the locations of the 96 subdural electrodes (blue dots), as well as the color-coded single-flash classification accuracy at each individual electrode.The traces at the bottom show the correlation between ECoG amplitude and the type of the stimulus (target/non-target) for cortical locations A–G.

Figure 6

Optimizing number of electrodes. The two figures show the relationship between the number of electrodes over visual cortex and accuracy (left) or bit rate (right) that this subject may achieve with these electrodes at one (blue circle), two (green triangle), and three (orange square) flash sequences. The subject may achieve a maximum of 100% classification accuracy at three flash sequences and four electrodes, and a maximum of 64 bits/min at two flash sequences and five electrodes.