(C)overt attention and visual speller design in an ERP-based brain-computer interface

Abstract

Background: In a visual oddball paradigm, attention to an event usually modulates the event-related potential (ERP). An ERP-based brain-computer interface (BCI) exploits this neural mechanism for communication. Hitherto, it was unclear to what extent the accuracy of such a BCI requires eye movements (overt attention) or whether it is also feasible for targets in the visual periphery (covert attention). Also unclear was how the visual design of the BCI can be improved to meet peculiarities of peripheral vision such as low spatial acuity and crowding.

Method: Healthy participants (N = 13) performed a copy-spelling task wherein they had to count target intensifications. EEG and eye movements were recorded concurrently. First, (c)overt attention was investigated by way of a target fixation condition and a central fixation condition. In the latter, participants had to fixate a dot in the center of the screen and allocate their attention to a target in the visual periphery. Second, the effect of visual speller layout was investigated by comparing the symbol Matrix to an ERP-based Hex-o-Spell, a two-levels speller consisting of six discs arranged on an invisible hexagon.

Results: We assessed counting errors, ERP amplitudes, and offline classification performance. There is an advantage (i.e., less errors, larger ERP amplitude modulation, better classification) of overt attention over covert attention, and there is also an advantage of the Hex-o-Spell over the Matrix. Using overt attention, P1, N1, P2, N2, and P3 components are enhanced by attention. Using covert attention, only N2 and P3 are enhanced for both spellers, and N1 and P2 are modulated when using the Hex-o-Spell but not when using the Matrix. Consequently, classifiers rely mainly on early evoked potentials in overt attention and on later cognitive components in covert attention.

Conclusions: Both overt and covert attention can be used to drive an ERP-based BCI, but performance is markedly lower for covert attention. The Hex-o-Spell outperforms the Matrix, especially when eye movements are not permitted, illustrating that performance can be increased if one accounts for peculiarities of peripheral vision.

Figure 1

Distribution of electrode sites on the scalp. Linked mastoids were used as reference. The three midline sites (magenta) and the four parieto-occipital sites (blue) refer to electrode subsets employed in the statistical analyses.

Figure 2

Screenshots of the two visual spellers. The current word was indicated in the box above the speller, and the current symbol was highlighted, (a) Symbol matrix. The column containing the target symbol "B" is intensified. (b) Hex-o-Spell, group level. The group containing the target symbol "B" (group "ABCDE") is intensified, (c) Transition phase. In a short animation, the symbols of the selected group are expanded onto the other discs. (d) Symbol level. The nontarget disc with the symbol "A" is intensified. The empty disc at the bottom is intended as a backdoor for returning to the group level in case the wrong group was selected.

Figure 3

Mean counting accuracy for the Matrix and the Hex-o-Spell in the two attention conditions. Error bars show 1 SEM. Counting accuracy was higher for the Hex-o-Spell than for the Matrix, and higher for overt attention than for covert attention. The performance drop from overt to covert attention was more severe for the Matrix than for the Hex-o-Spell.

Figure 4

Grand-average ERPs for each type of speller and each type of attention shown for 21 electrode sites. Responses to targets are given in magenta, responses to nontargets in black. Epochs in the range [-170, 670] are shown. In the overt attention condition, for both the Matrix and the Hex-o-Spell, there are two clear positive peaks at ≈ 250 ms (P2) and ≈ 400 ms (P3), with a broad scalp distribution centered at Cz. A positivity at ≈ 120 ms (P1) and a sharp negativity at ≈ 150 ms (N1) preponderates at parieto-occipital sites. Also, a N2 component (≈ 300 ms) is evident at these sites. In the covert attention condition, the response to targets versus nontargets is less differential. In comparison to the overt attention condition, amplitudes are smaller for targets and larger for nontargets. Furthermore, in this condition, P2 is pronounced only in the Hex-o-Spell at fronto-central electrode sites. The pattern of P3 amplitudes is similar to the one encountered in the overt attention condition. Note that the y-axis scaling is different for the two attention conditions.

Figure 5

Scalp distributions for the P1, N1, P2, N2, and P3 components. For visualization purposes, the intervals (depicted above the component labels) chosen for the components are slightly narrower than the intervals used in the ERP analysis. The shaded areas in the ERP plots correspond to the intervals for which topographies are shown underneath. The solid line refers to electrode Cz and the dotted line refers to the average of electrodes PO7 and PO8. In the overt attention condition, it is particularly N1 and P2 that yield high amplitudes for target intensifications. In the covert attention condition, P3 yields the highest amplitudes.

Figure 6

Mean amplitudes for the positive components P1, P2, and P3, and the negative components N1 and N2. Data is depicted separately for the Matrix and the Hex-o-Spell (colored in blue and magenta, respectively), and for overt and covert attention (left and right half of each plot, respectively). Error bars show 1 SEM. Note that the y-axis is reversed for negative components N1 and N2.

Figure 7

Results of the classification as a function of number of sequences. Thin colored lines indicate the performance of individual participants, the thick black line represents the mean in each subcondition. Accuracy approaches 100% using overt attention but is comparably low using covert attention. Hex-o-Spell outperforms the Matrix in both conditions.

Figure 8

Spatial and temporal distribution of discriminative information, (a) Classification errors obtained for each electrode separately are depicted as scalp topographies. (b) Classification error for a single temporal feature, a 40 ms averaging window, for different positions of the center of the window.