EEG-Based Music Emotion Prediction Using Supervised Feature Extraction for MIDI Generation

Abstract

Advancements in music emotion prediction are driving AI-driven algorithmic composition, enabling the generation of complex melodies. However, bridging neural and auditory domains remains challenging due to the semantic gap between brain-derived low-level features and high-level musical concepts, making alignment computationally demanding. This study proposes a deep learning framework for generating MIDI sequences aligned with labeled emotion predictions through supervised feature extraction from neural and auditory domains. EEGNet is employed to process neural data, while an autoencoder-based piano algorithm handles auditory data. To address modality heterogeneity, Centered Kernel Alignment is incorporated to enhance the separation of emotional states. Furthermore, regression between feature domains is applied to reduce intra-subject variability in extracted Electroencephalography (EEG) patterns, followed by the clustering of latent auditory representations into denser partitions to improve MIDI reconstruction quality. Using musical metrics, evaluation on real-world data shows that the proposed approach improves emotion classification (namely, between arousal and valence) and the system's ability to produce MIDI sequences that better preserve temporal alignment, tonal consistency, and structural integrity. Subject-specific analysis reveals that subjects with stronger imagery paradigms produced higher-quality MIDI outputs, as their neural patterns aligned more closely with the training data. In contrast, subjects with weaker performance exhibited auditory data that were less consistent.

Keywords: EEG; kernel methods; music emotion recognition; piano-roll algorithm.

Figure 1

Proposed deep learning framework for EEG-based emotion prediction using supervised feature extraction for MIDI generation. Stages: (i) segment-wise preprocessing; (ii) supervised deep feature extraction for emotion classification; and (iii) affective-based MIDI prediction and feature alignment.

Figure 2

Visualization of emotion labels and MIDI feature representations. (a) Emotion labels set by Subject $\#1$, where the x-axis represents arousal and the y-axis represents valence. (b) Two-dimensional t-SNE projection ($n\_components=2$, perplexity $=5$) of the piano-roll arrays, illustrating clustering of MIDI features based on emotion labels. Colors indicate the class of each audio stimulus. (c) Embedding space obtained from the bottleneck representation of the piano-roll autoencoder trained with CKA loss. Dots correspond to training data, while crosses (×) represent test data. Of note, the axes are resized to provide better visual perception of the plotted values.

Figure 3

Comparison between original MIDI (top) and reconstructed MIDI (bottom). Of note, the axes are resized to provide better visual perception of the plotted values.

Figure 4

Probability density functions (PDFs) of model characteristics for all subjects in fold 1. Each subfigure corresponds to a specific feature extracted from the MIDI data. (a) Feature 0: description of the feature (e.g., pitch range). (b) Feature 1: description of the feature (e.g., total used pitch). (c) Feature 2: description of the feature (e.g., average IOI). (d) Feature 3: description of the feature (e.g., pitch-class histogram).

Figure 5

Violin plots comparing metrics for the best- and worst-performing subjects.