Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2022 Jul 5:16:931085.
doi: 10.3389/fnhum.2022.931085. eCollection 2022.

Multi-Armed Bandits in Brain-Computer Interfaces

Frida Heskebeck  1 , Carolina Bergeling  2 , Bo Bernhardsson  1
Affiliations
Review

Multi-Armed Bandits in Brain-Computer Interfaces

Frida Heskebeck et al. Front Hum Neurosci. .

Abstract

The multi-armed bandit (MAB) problem models a decision-maker that optimizes its actions based on current and acquired new knowledge to maximize its reward. This type of online decision is prominent in many procedures of Brain-Computer Interfaces (BCIs) and MAB has previously been used to investigate, e.g., what mental commands to use to optimize BCI performance. However, MAB optimization in the context of BCI is still relatively unexplored, even though it has the potential to improve BCI performance during both calibration and real-time implementation. Therefore, this review aims to further describe the fruitful area of MABs to the BCI community. The review includes a background on MAB problems and standard solution methods, and interpretations related to BCI systems. Moreover, it includes state-of-the-art concepts of MAB in BCI and suggestions for future research.

Keywords: Brain-Computer Interface (BCI); calibration; multi-armed bandit (MAB); real-time optimization; reinforcement learning.

PubMed Disclaimer

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

    1. Abiri R., Borhani S., Sellers E. W., Jiang Y., Zhao X. (2019). A comprehensive review of EEG-based brain-computer interface paradigms. J. Neural Eng. 16:011001. 10.1088/1741-2552/aaf12e - DOI - PubMed
    1. Agrawal S., Goyal N. (2012). Analysis of thompson sampling for the multi-armed bandit problem, in Proceedings of the 25th Annual Conference on Learning Theory, Volume 23 of Proceedings of Machine Learning Research, eds Mannor S., Srebro N., Williamson R. C. (Edinburgh: ), 39.1–39.26.
    1. Alickovic E., Lunner T., Gustafsson F., Ljung L. (2019). A tutorial on auditory attention identification methods. Front. Neurosci. 13:153. 10.3389/fnins.2019.00153 - DOI - PMC - PubMed
    1. Auer P., Cesa-Bianchi N., Fischer P. (2002). Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47, 235–256. 10.1023/A:1013689704352 - DOI
    1. Besson L. (2018). SMPyBandits: An Open-Source Research Framework for Single and Multi-Players Multi-Arms Bandits (MAB) Algorithms in Python. Available online at: https://GitHub.com/SMPyBandits/SMPyBandits (accessed April 28, 2022).

LinkOut - more resources