EEG-to-EEG: Scalp-to-Intracranial EEG Translation Using a Combination of Variational Autoencoder and Generative Adversarial Networks

Abstract

A generative adversarial network (GAN) makes it possible to map a data sample from one domain to another one. It has extensively been employed in image-to-image and text-to image translation. We propose an EEG-to-EEG translation model to map the scalp-mounted EEG (scEEG) sensor signals to intracranial EEG (iEEG) sensor signals recorded by foramen ovale sensors inserted into the brain. The model is based on a GAN structure in which a conditional GAN (cGAN) is combined with a variational autoencoder (VAE), named as VAE-cGAN. scEEG sensors are plagued by noise and suffer from low resolution. On the other hand, iEEG sensor recordings enjoy high resolution. Here, we consider the task of mapping the scEEG sensor information to iEEG sensors to enhance the scEEG resolution. In this study, our EEG data contain epileptic interictal epileptiform discharges (IEDs). The identification of IEDs is crucial in clinical practice. Here, the proposed VAE-cGAN is firstly employed to map the scEEG to iEEG. Then, the IEDs are detected from the resulting iEEG. Our model achieves a classification accuracy of 76%, an increase of, respectively, 11%, 8%, and 3% over the previously proposed least-square regression, asymmetric autoencoder, and asymmetric-symmetric autoencoder mapping models.

Keywords: IED detection; generative adversarial networks; interictal epileptiform discharge; scalp-to-intracranial EEG translation; variational autoencoder.

Figure 1

The overviewof our proposed EEG-to-EEG network. $\mathbf{X}$ and $\mathbf{Y}$ are, respectively, the scEEG and iEEG. scEEG is fed to the encoder to be encoded to a latent space, which, along with scEEG, serves as an input to the generator. The generated data and the real iEEG are provided to the discriminator to be classified as real or fake.

Figure 2

The encoder network $\mathcal{E}$. The input data, $\mathbf{X}$, are scEEG. $C_e$ represents the coefficient of the number of filters for the Conv layer, and $I_E$ represents the total number of layers for the encoder. $\mathit{\mu }$ and $\mathit{\sigma }$ indicate, respectively, the mean and standard deviation of multivariate Gaussian distribution. $\mathbf{z}$ is the latent space.

Figure 3

(A) The SPADE block. In the SPADE block, the scEEG is first projected onto an embedding space and then convolved to produce the modulation parameters $\mathit{\gamma }$ and $\mathit{\beta }$. After normalizing the activation layer, $\mathcal{A}$, it is multiplied by $\mathit{\gamma }$ and added to $\mathit{\beta }$ element-wise. The SPADE is firstly proposed in [41]. (B) The SPADE ResNet consisting of two SAPDE blocks is followed by a Tanh activation and convolutional layers.

Figure 4

The generator network $\mathcal{G}$. scEGG, $\mathbf{X}$, and the latent space, $\mathbf{z}$, serve as input to the generator. L and M are, respectively, the number of time samples and channels of scEEG. $I_{up}$ is the number of upsampling or SPADE ResNet layers. The output, $\tilde{\mathbf{Y}}$, is the synthetic iEEG.

Figure 5

The proposed VAE-cGAN model for mapping the scEEG to iEEG. scEEG is encoded to a latent space. Then, the latent space alongside scEEG is fed to the generator to generate synthetic iEEG. The estimated iEEG and real iEEG are given to the discriminator to be classified as real or fake.

Figure 6

The diagram of the inter-subject classification approach.

Figure 7

(A) Samples of IEDs (top) and non-IEDs (bottom) averaged across all sensors are depicted. The scEEG, iEEG, and stimated iEEG generated by VAE-cGAN are illustrated. (B) Samples of IEDs (top) and non-IEDs (bottom) for a single sensor. The scEEG and estimated iEEG using VAE-cGAN are shown. In both (A,B), estimated iEEG follows the trend of iEEG in the IED samples. The IEDs start in point sample 32, and the sampling rate is 200 samples/s.

Figure 8

(A) Comparison of ACC, SEN, SPC, F1-S, and AUC metrics between our proposed VAE-cGAN method and the benchmarked LSR, AAE, ASAE, and cGAN methods in the inter-subject classification scenario. Notably, F1-S and AUC values were not available for the cGAN method. (B) The ROC curve provided by VAE-cGAN.