A brain-computer interface for the continuous, real-time monitoring of working memory load in real-world environments

Abstract

We developed a brain-computer interface (BCI) able to continuously monitor working memory (WM) load in real-time (considering the last 2.5 s of brain activity). The BCI is based on biomarkers derived from spectral properties of non-invasive electroencephalography (EEG), subsequently classified by a linear discriminant analysis classifier. The BCI was trained on a visual WM task, tested in a real-time visual WM task, and further validated in a real-time cross task (mental arithmetic). Throughout each trial of the cross task, subjects were given real or sham feedback about their WM load. At the end of the trial, subjects were asked whether the feedback provided was real or sham. The high rate of correct answers provided by the subjects validated not only the global behaviour of the WM-load feedback, but also its real-time dynamics. On average, subjects were able to provide a correct answer 82% of the time, with one subject having 100% accuracy. Possible cognitive and motor confounding factors were disentangled to support the claim that our EEG-based markers correspond indeed to WM.

Keywords: Electroencephalography; Machine learning; Neurophenomenology; Real-time brain–computer interfaces; Working memory.

Fig. 1

Representation of the low-WM-load task

Fig. 2

Confounders to be disentangled. The plane represents EEG biomarkers. Each ellipse is the set of biomarkers that change across conditions (for Task 1 and Task 2), or that change whenever the associated notion is present (WM, motor confounders and cognitive confounders). Area 1 represents the ideal WM markers. Area 2 represents (cognitive) activity necessary but not sufficient for WM. Area 3 represents potential motor confounders. Area 4, the remaining part of the crosshatched area, should be empty if all the potential confounders were correctly identified

Fig. 3

Electrode setup

Fig. 4

2-Parameter ROC curve for the Task 1 performed online. The curve has thickness because there are two parameters: the classification threshold and the required time of sustained activity. Each point represents a possible BCI design, and the corresponding specificity-sensitivity pair is the value for all the subjects

Fig. 5

Corrected ROC curve, after decorrelating markers of arousal

Fig. 6

Average over trials of the WMLE time evolution of a typical “good” subject. The first 2.5 s are not a reliable estimation, as the buffer requires 2.5 to be filled. The first 2.5 s of feedback were not displayed to the user

Fig. 7

Mean difference between high and low-WM-load conditions for relevant biomarkers. In light, values typically higher in high-WM-load condition. In dark, values typically lower in high-WM-load condition

Fig. 8

Design methodology