Development of Expert-Level Classification of Seizures and Rhythmic and Periodic Patterns During EEG Interpretation

Abstract

Background and objectives: Seizures (SZs) and other SZ-like patterns of brain activity can harm the brain and contribute to in-hospital death, particularly when prolonged. However, experts qualified to interpret EEG data are scarce. Prior attempts to automate this task have been limited by small or inadequately labeled samples and have not convincingly demonstrated generalizable expert-level performance. There exists a critical unmet need for an automated method to classify SZs and other SZ-like events with expert-level reliability. This study was conducted to develop and validate a computer algorithm that matches the reliability and accuracy of experts in identifying SZs and SZ-like events, known as "ictal-interictal-injury continuum" (IIIC) patterns on EEG, including SZs, lateralized and generalized periodic discharges (LPD, GPD), and lateralized and generalized rhythmic delta activity (LRDA, GRDA), and in differentiating these patterns from non-IIIC patterns.

Methods: We used 6,095 scalp EEGs from 2,711 patients with and without IIIC events to train a deep neural network, SPaRCNet, to perform IIIC event classification. Independent training and test data sets were generated from 50,697 EEG segments, independently annotated by 20 fellowship-trained neurophysiologists. We assessed whether SPaRCNet performs at or above the sensitivity, specificity, precision, and calibration of fellowship-trained neurophysiologists for identifying IIIC events. Statistical performance was assessed by the calibration index and by the percentage of experts whose operating points were below the model's receiver operating characteristic curves (ROCs) and precision recall curves (PRCs) for the 6 pattern classes.

Results: SPaRCNet matches or exceeds most experts in classifying IIIC events based on both calibration and discrimination metrics. For SZ, LPD, GPD, LRDA, GRDA, and "other" classes, SPaRCNet exceeds the following percentages of 20 experts-ROC: 45%, 20%, 50%, 75%, 55%, and 40%; PRC: 50%, 35%, 50%, 90%, 70%, and 45%; and calibration: 95%, 100%, 95%, 100%, 100%, and 80%, respectively.

Discussion: SPaRCNet is the first algorithm to match expert performance in detecting SZs and other SZ-like events in a representative sample of EEGs. With further development, SPaRCNet may thus be a valuable tool for an expedited review of EEGs.

Classification of evidence: This study provides Class II evidence that among patients with epilepsy or critical illness undergoing EEG monitoring, SPaRCNet can differentiate (IIIC) patterns from non-IIIC events and expert neurophysiologists.

Figure 1. ROC Curves

Solid curves are median ROC curves that show model performance; shading indicates 95% confidence bands. Expert operating points (x, y) on the ROC curve are shown as solid circles with (x, y) = (false-positive rate [FPR, aka 1 − specificity], true-positive rate [TPR, aka sensitivity]). Markers are colored in black when they lie above the median ROC curve of the model (better than model performance) and in gray when they lie below (inferior to model performance). EUROC = % of experts under the ROC curve; GPD = generalized periodic discharge; GRDA = generalized rhythmic delta activity; LPD = lateralized periodic discharge; LRDA = lateralized rhythmic delta activity; PPV = positive predicted value; ROC = receiver operating characteristic.

Figure 2. PR Curves

Solid curves are median PR curves that show model performance; shading indicates 95% confidence bands. Expert operating points (x, y) on the PR curve are shown as solid triangles with (x, y) = (TPR, precision [aka positive predictive value (PPV)]). Markers are colored in black when they lie above the median PR curve of the model (better than model performance) and in gray when they lie below (inferior to model performance). EUPRC = % of experts under the PR curve; GPD = generalized periodic discharge; GRDA = generalized rhythmic delta activity; LPD = lateralized periodic discharge; LRDA = lateralized rhythmic delta activity; PR = precision recall.

Figure 3. Maps of the Ictal-Interictal-Injury Continuum Learned by SPaRCNet

Two-dimensional coordinates are calculated by an algorithm (UMAP) such that patterns assigned similar probabilities for each class by the model are near each other in the map. The map learned by SparCNet (model) forms a “starfish” pattern, with the 5 IIIC patterns (SZ, LPD, GPD, LRDA, and GRDA) emanating as arms from a central region containing non-IIIC patterns. The coloring of the map indicates the model's classification decisions and closely matches the pattern obtained by overlaying expert-consensus labels (human). Model uncertainty (uncertainty), indicating the degree to which the model assigns similar probabilities to multiple patterns, is greatest near the central region and decreases toward the tips of the “starfish” arms. The probability that an EEG segment represents a seizure or any one of the 4 most highly epileptiform patterns (the sum of the probabilities of SZ, LPD, GPD, or LRDA is shown in SZ burden and IIIC burden). GPD = generalized periodic discharge; GRDA = generalized rhythmic delta activity; IIIC = ictal-interictal-injury continuum; LPD = lateralized periodic discharge; LRDA = lateralized rhythmic delta activity; SZ = seizure.

Figure 4. Examples of Smooth Pattern Transition for SZ (A) and LPD (B)

Samples are selected at different levels of model uncertainty ranging from the “starfish” arm tips toward the central area. IIIC patterns transition smoothly between distinct prototype patterns at the starfish arm tips into less distinct patterns near the body, lending credence to the concept of a “continuum” between ictal and interictal EEG patterns. IIIC = ictal-interictal-injury continuum; LPD = lateralized periodic discharge; SZ = seizure.

Figure 5. Examples of Smooth Pattern Transition for GPD (A) and LRDA (B)

Samples are selected at different levels of model uncertainty ranging from the “starfish” arm tips toward the central area. GPD = generalized periodic discharge; LRDA = lateralized rhythmic delta activity.

Figure 6. Examples of Smooth Pattern Transition for GRDA (A) and “Other” (B)

Samples are selected at different levels of model uncertainty ranging from the “starfish” arm tips toward the central area. GRDA = generalized rhythmic delta activity.