Prospective evaluation of interrater agreement between EEG technologists and neurophysiologists

Abstract

We aim to prospectively investigate, in a large and heterogeneous population, the electroencephalogram (EEG)-reading performances of EEG technologists. A total of 8 EEG technologists and 5 certified neurophysiologists independently analyzed 20-min EEG recordings. Interrater agreement (IRA) for predefined EEG pattern identification between EEG technologists and neurophysiologits was assessed using percentage of agreement (PA) and Gwet-AC1. Among 1528 EEG recordings, the PA [95% confidence interval] and interrater agreement (IRA, AC1) values were as follows: status epilepticus (SE) and seizures, 97% [96-98%], AC1 kappa = 0.97; interictal epileptiform discharges, 78% [76-80%], AC1 = 0.63; and conclusion dichotomized as "normal" versus "pathological", 83.6% [82-86%], AC1 = 0.71. EEG technologists identified SE and seizures with 99% [98-99%] negative predictive value, whereas the positive predictive values (PPVs) were 48% [34-62%] and 35% [20-53%], respectively. The PPV for normal EEGs was 72% [68-76%]. SE and seizure detection were impaired in poorly cooperating patients (SE and seizures; p < 0.001), intubated and older patients (SE; p < 0.001), and confirmed epilepsy patients (seizures; p = 0.004). EEG technologists identified ictal features with few false negatives but high false positives, and identified normal EEGs with good PPV. The absence of ictal features reported by EEG technologists can be reassuring; however, EEG traces should be reviewed by neurophysiologists before taking action.

Figure 1

Percentages of true positives (in dark green; EEG patterns coded “yes” by both EEG technologists and neurophysiologists), true negatives (in light green; EEG patterns coded “no” by both EEG technologists and neurophysiologists), false positives (in orange; exams coded “yes” by EEG technologists and “no” by neurophysiologists), and false negatives (in red; exams coded “no” by EEG technologists and “yes” by neurophysiologists). FN False negative, FP false positive, TP true positive, TN true negative, IEDS interictal epileptiform discharges, PDR posterior dominant rhythm, SE status epilepticus, Sz seizures.