Brain-computer-interface-driven artistic expression: real-time cognitive visualization in the pangolin scales animatronic dress and screen dress

Abstract

This paper explores the intersection of brain-computer interfaces (BCIs) and artistic expression, showcasing two innovative projects that merge neuroscience with interactive wearable technology. BCIs, traditionally applied in clinical settings, have expanded into creative domains, enabling real-time monitoring and representation of cognitive states. The first project showcases a low-channel BCI Screen Dress, utilizing a 4-channel electroencephalography (EEG) headband to extract an engagement biomarker. The engagement is visualized through animated eyes on small screens embedded in a 3D-printed dress, which dynamically responds to the wearer's cognitive state. This system offers an accessible approach to cognitive visualization, leveraging real-time engagement estimation and demonstrating the effectiveness of low-channel BCIs in artistic applications. In contrast, the second project involves an ultra-high-density EEG (uHD EEG) system integrated into an animatronic dress inspired by pangolin scales. The uHD EEG system drives physical movements and lighting, visually and kinetically expressing different EEG frequency bands. Results show that both projects have successfully transformed brain signals into interactive, wearable art, offering a multisensory experience for both wearers and audiences. These projects highlight the vast potential of BCIs beyond traditional clinical applications, extending into fields such as entertainment, fashion, and education. These innovative wearable systems underscore the ability of BCIs to expand the boundaries of creative expression, turning the wearer's cognitive processes into art. The combination of neuroscience and fashion tech, from simplified EEG headsets to uHD EEG systems, demonstrates the scalability of BCI applications in artistic domains.

Keywords: 3D-print; BCI; animatronic; art; engagement; fashion-tech; uHD EEG.

Figure 1

(A) Project 1: screen dress (©Anouk Wipprecht), (B) Project 2: pangolin scales dress (©Yanni de Melo).

Figure 2

Unicorn BCI Core-4: 4-channel EEG headband device utilizing dry electrode technology.

Figure 3

Example of the d2 test performed during the training session [adapted from Natalizio et al., 2024].

Figure 4

Schematic overview of the system setup: featuring the 4-channel EEG headband for data acquisition, signal processing on PC, and the Hyperpixel for visualization embedded in the Screen Dress.

Figure 5

uHD EEG system g.Pangolin. (A) electrode grids, pre-amplifier, and medical adhesives, (B) connectorbox.

Figure 6

Electrode distribution of the uHD EEG system (small black empty circles) compared to the standard 10-20-system (dark gray filled circles), 10–10 system (light gray filled circles), and the extended 10–10 system (light gray empty circles).

Figure 7

(A) Hardware board, including the Arduino nano μC (1) and 2x motor driver boards (2), powered via battery pack (3) and connected via USB to the control PC (4); (B) Servo motor and the LED pixel mounted on the dress components.

Figure 8

Signal Processing pipeline from the BCI system used for controlling the dress, from preprocessing the raw EEG through feature extraction to the final class decision.

Figure 9

(A) Brain model with the uHD grids and functional areas; (B) functional brain areas marked according to the Desikan-Killiany atlas as described by Desikan et al. (2006).

Figure 10

Results of the evaluation run (A) Score values over time for engagement condition (red) and rest condition (blue), (B) score values for each class, (C) 3D-eye animation reaction according to score values and the corresponding condition.

Figure 11

The three dress states with the dress mounted on a mannequin. (A) Theta–meditation, creativity (purple), (B) Alpha–relaxed, awake (blue), (C) Beta–alertness, stress (white).

Figure 12

Pangolin Scales EEG electrode grids and interactive dress worn by the model (©Florian Voggeneder).