User's Self-Prediction of Performance in Motor Imagery Brain-Computer Interface

Abstract

Performance variation is a critical issue in motor imagery brain-computer interface (MI-BCI), and various neurophysiological, psychological, and anatomical correlates have been reported in the literature. Although the main aim of such studies is to predict MI-BCI performance for the prescreening of poor performers, studies which focus on the user's sense of the motor imagery process and directly estimate MI-BCI performance through the user's self-prediction are lacking. In this study, we first test each user's self-prediction idea regarding motor imagery experimental datasets. Fifty-two subjects participated in a classical, two-class motor imagery experiment and were asked to evaluate their easiness with motor imagery and to predict their own MI-BCI performance. During the motor imagery experiment, an electroencephalogram (EEG) was recorded; however, no feedback on motor imagery was given to subjects. From EEG recordings, the offline classification accuracy was estimated and compared with several questionnaire scores of subjects, as well as with each subject's self-prediction of MI-BCI performance. The subjects' performance predictions during motor imagery task showed a high positive correlation (r = 0.64, p < 0.01). Interestingly, it was observed that the self-prediction became more accurate as the subjects conducted more motor imagery tasks in the Correlation coefficient (pre-task to 2nd run: r = 0.02 to r = 0.54, p < 0.01) and root mean square error (pre-task to 3rd run: 17.7% to 10%, p < 0.01). We demonstrated that subjects may accurately predict their MI-BCI performance even without feedback information. This implies that the human brain is an active learning system and, by self-experiencing the endogenous motor imagery process, it can sense and adopt the quality of the process. Thus, it is believed that users may be able to predict MI-BCI performance and results may contribute to a better understanding of low performance and advancing BCI.

Keywords: BCI; BCI-illiteracy; motor imagery; performance variation; prediction.

FIGURE 1

Overall experiment design. Before the 1st run, personal information was collected, and subjects practiced motor imagery and predicted their classification accuracy. Then, resting state EEG was recorded. Between tasks, the follow-up questions were asked to collect the self-assessed condition levels and motor imagery related scores, including prediction of BCI performance.

FIGURE 2

Accuracy comparisons between different groups. Mean accuracy with standard deviations are presented, and the total number of subjects within each group is noted on the bottom of each figure: (A) Sex, (B) Age, (C) Coffee, (D) Alcohol, and (E) Cigarettes. Figure (F) notes the correlation coefficients between physical/mental scores (FDR-corrected).

FIGURE 3

Correlations with actual classification accuracy. (A) The relationships between measures are presented with correlation coefficients. The non-significant correlation (p > 0.05) and significant correlation (p < 0.05) are presented with dotted and solid lines, respectively. Line width represents the strength of correlation. (B) Dots in each figure represent subjects, and the black line is the linear regression line to data points. Statistical significance is marked with one star (p < 0.05) or double stars (p < 0.01) on the right bottom of each figure (p-values were FDR corrected).

FIGURE 4

Self-prediction comparison across task runs. (A) The evolution of accuracy and self-prediction from pre-task to the 5th run. (B) Correlation coefficients between self-predicted performance and actual classification performance are presented across pre-task and the 1st to 5th runs. (C) Corresponding Root Mean Square Error between the predicted performance and actual classification performance are presented. Statistical lines are marked with the dotted (p = 0.01) and the dashed (p = 0.05) lines.

FIGURE 5

Correlation coefficient comparison across different methods. For the neurophysiological predictor, the value from the study by Ahn et al. (2013b) was adopted.

FIGURE 6

Evolutions of performance and prediction. Four representative figures show how the performances and predictions evolve across runs. Classification accuracy (CA) from all the trials (5 runs), classification accuracy (CA run) at each run, and accuracy prediction (AP) at each run are presented.