Classifier-based latency estimation: a novel way to estimate and predict BCI accuracy

Abstract

Objective: Brain-computer interfaces (BCIs) that detect event-related potentials (ERPs) rely on classification schemes that are vulnerable to latency jitter, a phenomenon known to occur with ERPs such as the P300 response. The objective of this work was to investigate the role that latency jitter plays in BCI classification.

Approach: We developed a novel method, classifier-based latency estimation (CBLE), based on a generalization of Woody filtering. The technique works by presenting the time-shifted data to the classifier, and using the time shift that corresponds to the maximal classifier score.

Main results: The variance of CBLE estimates correlates significantly (p < 10(-42)) with BCI accuracy in the Farwell-Donchin BCI paradigm. Additionally, CBLE predicts same-day accuracy, even from small datasets or datasets that have already been used for classifier training, better than the accuracy on the small dataset (p < 0.05). The technique should be relatively classifier-independent, and the results were confirmed on two linear classifiers.

Significance: The results suggest that latency jitter may be an important cause of poor BCI performance, and methods that correct for latency jitter may improve that performance. CBLE can also be used to decrease the amount of data needed for accuracy estimation, allowing research on effects with shorter timescales.

Figure 1

Accuracy plotted against classifier-based latency jitter estimates (CBLE) using two classifiers. Left: results from a classifier using least-squares (LS) regression; right: the popular step-wise linear discriminant analysis (SWLDA) classifier.

Figure 2

Accuracy vs. variance of CBLE by participant. The data used are the same as in the left panel of Figure 1.

Figure 3

Predicting first-day accuracy based on a 5-character dataset, by classifier type and estimation method. Left: least-squares classification (LS); right: classification using step-wise linear discriminant analysis (SWLDA). Top: prediction based on 5-character accuracy; bottom: prediction based on classifier-based latency jitter estimates (CBLE).

Figure 4

Predicting first-day accuracy based on training data, by classifier type and estimation method. Left: least-squares classification (LS); right: classification using step-wise linear discriminant analysis (SWLDA). Top: prediction based on training accuracy; middle: prediction based on leave-one-out cross-validation accuracy on training data; bottom: prediction based on classifier-based latency jitter estimates (CBLE).