Implementation of an Online Auditory Attention Detection Model with Electroencephalography in a Dichotomous Listening Experiment

Abstract

Auditory attention detection (AAD) is the tracking of a sound source to which a listener is attending based on neural signals. Despite expectation for the applicability of AAD in real-life, most AAD research has been conducted on recorded electroencephalograms (EEGs), which is far from online implementation. In the present study, we attempted to propose an online AAD model and to implement it on a streaming EEG. The proposed model was devised by introducing a sliding window into the linear decoder model and was simulated using two datasets obtained from separate experiments to evaluate the feasibility. After simulation, the online model was constructed and evaluated based on the streaming EEG of an individual, acquired during a dichotomous listening experiment. Our model was able to detect the transient direction of a participant's attention on the order of one second during the experiment and showed up to 70% average detection accuracy. We expect that the proposed online model could be applied to develop adaptive hearing aids or neurofeedback training for auditory attention and speech perception.

Keywords: dichotomous listening; electroencephalography; linear decoder model; online auditory attention detection; sliding window.

Figure 1

Online decoder model. (a) Construction of an online decoder model. Online decoder model ($D$) is the average of the individual decoders ($D_j^i$) estimated with snippets of EEG signal ($R_j^i$) and corresponding speech signal ($S_j^i$). $i$ and $j$ are the index of trial and snippet, respectively. (b) Implementation of the online decoder model in a dichotic listening scenario. (c) Detection of the direction of auditory attention based on the correlation between reconstructed and actual speech envelopes. An example of online detection results is plotted on the right.

Figure 2

Online auditory attention detection (AAD) experiment. (a) An illustration of the experimental procedure. (b) Online data processing pipeline for model construction and testing.

Figure 3

Online AAD simulation results. (a) Heatmaps showing an overview of changes in average detection accuracy according to parameter values for two datasets. Selected parameter values for the online AAD model are colored red. (b) Boxplots showing the effects of each parameter on detection accuracy. For each boxplot, the detection accuracies presented in (a) are collapsed to each parameter. The edges of each box denote the 25th and 75th quantiles, and the middle line in each box refers to the median. Again, the selected value of each parameter is colored red. (c) Online AAD simulation results from both datasets applying the selected parameters. A black line on each bar denotes ± 1 standard deviation.

Figure 4

Online AAD experiment results. (a,b) Online AAD results of four attention-fixed trials and four attention-switching trials. (c) The average detection accuracy of all the participants for the 16 test trials, the attention-fixed trials (12 trials), and the attention-switching trials (4 trials). A black line on each bar denotes ± 1 standard deviation. A dashed gray horizontal line on the bottom of each plot signifies the chance level.