A Brain-Controlled Quadruped Robot: A Proof-of-Concept Demonstration

Abstract

Coupling brain-computer interfaces (BCIs) and robotic systems in the future can enable seamless personal assistant systems in everyday life, with the requests that can be performed in a discrete manner, using one's brain activity only. These types of systems might be of a particular interest for people with locked-in syndrome (LIS) or amyotrophic lateral sclerosis (ALS) because they can benefit from communicating with robotic assistants using brain sensing interfaces. In this proof-of-concept work, we explored how a wireless and wearable BCI device can control a quadruped robot-Boston Dynamics' Spot. The device measures the user's electroencephalography (EEG) and electrooculography (EOG) activity of the user from the electrodes embedded in the glasses' frame. The user responds to a series of questions with YES/NO answers by performing a brain-teaser activity of mental calculus. Each question-answer pair has a pre-configured set of actions for Spot. For instance, Spot was prompted to walk across a room, pick up an object, and retrieve it for the user (i.e., bring a bottle of water) when a sequence resolved to a YES response. Our system achieved at a success rate of 83.4%. To the best of our knowledge, this is the first integration of wireless, non-visual-based BCI systems with Spot in the context of personal assistant use cases. While this BCI quadruped robot system is an early prototype, future iterations may embody friendly and intuitive cues similar to regular service dogs. As such, this project aims to pave a path towards future developments in modern day personal assistant robots powered by wireless and wearable BCI systems in everyday living conditions.

Keywords: artificial intelligence; brain–computer interfaces; electroencephalography (EEG); electrooculography (EOG); personal assistance; robots; wearables.

Figure 1

The overview of the state of the art in BCI and robotics as of 2023. Selected papers are featured, representative of the directions of the research, including (A) [29] and (F) [8]—the implants used to control limbs and exoskeleton using MI BCI paradigm; (E) [30]—the most recent paper to date, featuring a hybrid MI + SSVEP + EMG system; (B) [16], (D) [31], and (G) [32], all using the same cap for different robotic use cases as well as the MI BCI paradigm; (C) [33] featuring the P300 interface; and finally, (H) [34], the only paper to the best of our knowledge featuring a quadruped robot and using the SSVEP BCI paradigm. All users trained on these aforementioned systems also required a visual-based training protocol. The figure also illustrates the challenges of portability and comfort for the users who would wear BCIs, with our proposed solution in this paper (I) being the only truly wearable form-factor with a setup time under 2 min.

Figure 2

The architecture of the Ddog system.

Figure 3

The overview of the entire system.

Figure 4

Visual representation of Cloud B and Cloud D.

Figure 5

Arm and gripper for the Spot robot by Boston Dynamics.

Figure 6

A person wearing AttentivU glasses about to perform a mental task of calculations in order to send Spot from the ‘living room’ space to the ‘kitchen space’.

Figure 7

AttentivU glasses (left) and montage of EEG electrode locations (right). AttentivU glasses consisting of 2 EEG channels, TP9, and TP10, as well as a reference electrode at Fpz. It additionally has EOG channels and built-in audio and haptic feedback.

Figure 8

Task structure for a run. Here, task and rest were of a duration of 2 min each, whereas calibration and break were 40 s each.

Figure 9

The above curve is the alpha/delta ratio, which is used to classify the MA task. The changes in this curve are used to determine when the subject changed their mental state.

Figure 10

BioSignal application’s UI for a YES/NO choice. From left to right: countdown to select YES as a response; countdown to select NO as a response. Response YES is an output of the system.