The MindGomoku: An Online P300 BCI Game Based on Bayesian Deep Learning

Abstract

In addition to helping develop products that aid the disabled, brain-computer interface (BCI) technology can also become a modality of entertainment for all people. However, most BCI games cannot be widely promoted due to the poor control performance or because they easily cause fatigue. In this paper, we propose a P300 brain-computer-interface game (MindGomoku) to explore a feasible and natural way to play games by using electroencephalogram (EEG) signals in a practical environment. The novelty of this research is reflected in integrating the characteristics of game rules and the BCI system when designing BCI games and paradigms. Moreover, a simplified Bayesian convolutional neural network (SBCNN) algorithm is introduced to achieve high accuracy on limited training samples. To prove the reliability of the proposed algorithm and system control, 10 subjects were selected to participate in two online control experiments. The experimental results showed that all subjects successfully completed the game control with an average accuracy of 90.7% and played the MindGomoku an average of more than 11 min. These findings fully demonstrate the stability and effectiveness of the proposed system. This BCI system not only provides a form of entertainment for users, particularly the disabled, but also provides more possibilities for games.

Keywords: BCI game; Bayesian deep learning; P300; brain–computer interface (BCI); electroencephalogram (EEG).

Figure 1

The framework of the BCI game, which contains three subsystems as follows: (a) data acquisition, (b) data processing, and (c) visual and game terminal. The data acquisition part records the electroencephalogram (EEG) signal. After the signal is preprocessed, the data processing part can be divided into two steps, off-line classifier training and online classifier testing. The visual and game terminal provides users visual stimuli after updating the stimulus strategy and provides corresponding visual feedback (output coordinates).

Figure 2

An illustration of the MindGomoku. (a) The graphical user interface of the MindGomoku. (b) The online interactive paradigm of the MindGomoku. It takes two steps to select the coordinate of the red star. According to its location, the user should select the character M in the first-level interface and then select the character 6 in the second-level interface. The two selections can determine a coordinate, at which the system will present a piece in the Go board as feedback.

Figure 3

Architecture of a simplified Bayesian convolutional neural network (SBCNN) for feature extraction and classification.

Figure 4

Each subject’s off-line accuracies and all subjects’ average accuracies in different repeats in experiment I. The vertical axis corresponds to the accuracy rate, and the horizontal axis corresponds to the number of repeats. The color curves represent classification accuracy rates in the first sub-trial, the second sub-trial, and the complete trial.

Figure 5

The mean ± standard classification accuracies of all 10 subjects in different repeats during experiment I: (a) in the first sub-trial, (b) in the second sub-trial, and (c) in the complete trial.

Figure 5

The mean ± standard classification accuracies of all 10 subjects in different repeats during experiment I: (a) in the first sub-trial, (b) in the second sub-trial, and (c) in the complete trial.

Figure 6

The average target recognition accuracy of all 10 subjects in different repeats trained on half training sets. The color curves represent classification accuracy rates in the SBCNN, BN3, EEGNET and CNN-1. (a) in the first sub-trial, (b) in the second sub-trial, and (c) in the complete trial.

Figure 6

The average target recognition accuracy of all 10 subjects in different repeats trained on half training sets. The color curves represent classification accuracy rates in the SBCNN, BN3, EEGNET and CNN-1. (a) in the first sub-trial, (b) in the second sub-trial, and (c) in the complete trial.