Utilizing Pretrained Vision Transformers and Large Language Models for Epileptic Seizure Prediction

Abstract

Repeated unprovoked seizures are a major source of concern for people with epilepsy. Predicting seizures before they occur is of interest to both machine-learning scientists as well as clinicians, and is an active area of research. The variability of EEG sensors, type of seizures, and specialized knowledge required for annotating the data complicates the large-scale annotation process essential for supervised predictive models. To address these challenges, we propose the use of Vision Transformers (ViTs) and Large Language Models (LLMs) that were originally trained on publicly available image or text data. Our work leverages these pre-trained models by refining the input, embedding, and classification layers in a minimalistic fashion to predict seizures. Our results demonstrate that LLMs outperforms the ViTs in patient-independent seizure prediction achieving a sensitivity of 79.02% which is 8% higher compared to ViTs and about 12% higher compared to a custom-designed ResNet-based model. Our work demonstrates the successful feasibility of pre-trained models for seizure prediction with its potential for improving the quality of life of people with epilepsy. Our code and related materials are available open-source at: https://github.com/pcdslab/UtilLLM_EPS/.

Keywords: Electroencephalography (EEG); Epilepsy; Large Language Model (LLM); Seizure Prediction; Vision Transformer (ViT).

Fig. 1.

Illustration of the phases in which data is augmented to make it suitable for being input into a ViT.

Fig. 2.

Illustration of the different stages of ViT-2 with the re-trained layers encapsulated by dotted rectangles.