Diagnostic accuracy between readers for identifying electrographic seizures in critically ill adults

Abstract

Objective: Electrographic seizures in critically ill patients are often equivocal. In this study, we sought to determine the diagnostic accuracy of electrographic seizure annotation in adult intensive care units (ICUs) and to identify affecting factors.

Methods: To investigate diagnostic accuracy, interreader agreement (IRA) measures were derived from 5,769 unequivocal and 6,263 equivocal seizure annotations by five experienced electroencephalogram (EEG) readers after reviewing 74 days of EEGs from 50 adult ICU patients. Factors including seizure equivocality (unequivocal vs. equivocal) and laterality (generalized, partial, or bilaterally independent), cyclicity (cyclic vs. noncyclic), persistency (occurrence of status epilepticus), and patient consciousness level (coma vs. noncoma) were further investigated for their influence on IRA measures.

Results: On average, 70% of seizures marked by a reference reader overlapped, at least in part, with those marked by a test reader (any-overlap sensitivity, AO-Sn). Agreed seizure duration between reader pairs (overlap-integral sensitivity, OI-Sn) was 62%, while agreed nonseizure duration (overlap-integral specificity, OI-Sp) was 99%. A test reader would annotate one additional seizure not overlapping with a reference reader's annotation in every 11.7 h of EEG, that is, the false-positive rate (FPR) was 0.0854/h. Classifying seizure patterns into unequivocal and equivocal improved specificity and FPR (unequivocal patterns) but compromised sensitivity only for equivocal patterns. Sensitivity of all and unequivocal annotations was higher for patients with status epilepticus. Specificity was higher for partial than for bilaterally independent unequivocal seizure patterns, and lower for cyclic all seizure patterns.

Significance: Diagnosing electrographic seizures in critically ill adults is highly specific and moderately sensitive. Improved criteria for diagnosing electrographic seizures in the ICU are needed.

Keywords: Continuous EEG monitoring; Critical care; Epilepsy; Equivocal seizures; Interreader agreement.

Figure 1

OI‐Sn versus OI‐Sp scattergrams between each reader pair. Data shown are 5 × 4 × 50 matrices used in the statistical analysis for this study. Each data point represents OI‐Sn (y coordinate) and OI‐Sp (x coordinate) of each patient (n = 50). Each row represents OI‐Sn and OI‐Sp value pairs of a test reader (R1–R5 in row labels) against each reference reader (R1–R5 in column labels).

Figure 2

Diagnostic accuracy of ICU seizure detection was highly specific (OI‐Sn of 99% and FPR of 0.085/h) but moderately sensitive (AO‐Sn of 70%, OI‐Sn of 62%), with higher specificity for both unequivocal and equivocal patterns and lower sensitivity for equivocal patterns. The sensitivity‐FPR plot for any‐overlap measures (A) and sensitivity‐specificity plot for overlap‐integral measures (B) demonstrate that, compared to annotation of all seizures (All, black data points), classification of seizure patterns as unequivocal (Uneq, red data points) or equivocal ones (Eq, pink data points) based on ACNS guidelines improved specificity and FPR for unequivocal patterns and lowered sensitivity (both AO‐Sn and OI‐Sn), but improved specificity, for equivocal patterns. (Values are means, with error bars representing 95% confidence intervals.)

Figure 3

EEG of an exemplary equivocal seizure disagreed upon by four of the five readers. Only the left parasagittal chain in bipolar longitudinal montage was shown because other regions did not show suspicious patterns. The onset and offset (red vertical lines) were marked by one reader and stated in the EEG report. The evolving pattern in the left parasagittal region (F3 C3 P3 O1 electrode) was composed of a blunt, <3‐Hz, minimally evolving pattern, with high‐frequency artifacts at C3, where a burr hole for evacuation of a subdural hematoma was located. This pattern gradually became shorter and disappeared within 3 h after intravenous administration of 500 mg phenytoin.

Figure 4

Diagnostic accuracy was affected by seizure laterality, status epilepticus, and seizure cyclicity. (A) For laterality of unequivocal seizure patterns (Uneq), annotation of bilaterally independent seizure patterns (BI) were lower in OI‐Sp than those of partial seizure patterns (P), but not for generalized seizure patterns (Gen). (B) For patients with status epilepticus (SE, green horizontal hyphens), seizure annotations had higher AO‐Sn and OI‐Sn for all seizure patterns (All) and unequivocal patterns (Uneq). (C) For cyclicity, annotation of EEGs containing cyclic seizures (cyclic, green horizontal hyphens) was higher in FPR for all and unequivocal patterns than for those without cyclic seizures (noncyclic, brown horizontal hyphens) for all annotation types. (D) Cyclicity also lowered OI‐Sp for all seizure patterns. (Data are displayed as mean [horizontal hyphens], with error bars representing 95% confidence intervals; *p < 0.05, Bonferroni correction.)