Assessment of a scalp EEG-based automated seizure detection system

Abstract

Objective: The purpose of this study was to evaluate and validate an offline, automated scalp EEG-based seizure detection system and to compare its performance to commercially available seizure detection software.

Methods: The test seizure detection system, IdentEvent™, was developed to enhance the efficiency of post-hoc long-term EEG review in epilepsy monitoring units. It translates multi-channel scalp EEG signals into multiple EEG descriptors and recognizes ictal EEG patterns. Detection criteria and thresholds were optimized in 47 long-term scalp EEG recordings selected for training (47 subjects, ∼3653h with 141 seizures). The detection performance of IdentEvent was evaluated using a separate test dataset consisting of 436 EEG segments obtained from 55 subjects (∼1200h with 146 seizures). Each of the test EEG segments was reviewed by three independent epileptologists and the presence or absence of seizures in each epoch was determined by majority rule. Seizure detection sensitivity and false detection rate were calculated for IdentEvent as well as for the comparable detection software (Persyst's Reveal®, version 2008.03.13, with three parameter settings). Bootstrap re-sampling was applied to establish the 95% confidence intervals of the estimates and for the performance comparison between two detection algorithms.

Results: The overall detection sensitivity of IdentEvent was 79.5% with a false detection rate (FDR) of 2 per 24h, whereas the comparison system had 80.8%, 76%, and 74% sensitivity using its three detection thresholds (perception score) with FDRs of 13, 8, and 6 per 24h, respectively. Bootstrap 95% confidence intervals of the performance difference revealed that the two detection systems had comparable detection sensitivity, but IdentEvent generated a significantly (p<0.05) smaller FDR.

Conclusions: The study validates the performance of the IdentEvent™ seizure detection system.

Significance: With comparable detection sensitivity, an improved false detection rate makes the automated seizure detection software more useful in clinical practice.

Figure 1

An example of a PMRS curve before, during, and after a seizure (between two vertical dotted red lines). PMRS values drop significantly during the ictal period compared to other periods.

Figure 2

Number of sampled “seizure” and “non-seizure” segments for all 55 test subjects.

Figure 3

Length of sampled EEG segments for all 55 test subjects.

Figure 4

Adjusted number of seizures (randomly down-sampled when the # of seizures was >5) for all 55 test subjects.

Figure 5

A flow chart of the IdentEvent seizure detection system.

Figure 6

IdentEvent’s detection sensitivity for each individual subject. The independent EEG reviewers identified at least one seizure in 46 subjects.

Figure 7

IdentEvent’s false detection rate per 24 hours for all test subjects.

Figure 8

Bootstrap distribution of the sensitivity difference (sensitivity_IdentEvent – sensitivity_Reveal). Reveal perception scores were set to 0.5, 0.8, and 0.9.

Figure 9

Bootstrap distribution of the false detection rate difference (false detection rate_IdentEvent – false detection rste_Reveal). The Reveal perception score was set to 0.5, 0.8, and 0.9.

Figure 10

PMRS time series 15 minutes before and after a detection. Top: Detection of a seizure. The PMRS values during the entire ictal period are significantly lower than other periods such that it creates a concave. Bottom: Detection of a sudden sleep stage change. PMRS exhibits a sharp drop during the change, but the values quickly resume to normal after the change.