Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2017 Mar;34(2):168-173.
doi: 10.1097/WNP.0000000000000344.

Characteristics of EEG Interpreters Associated With Higher Interrater Agreement

Jonathan J Halford  1 , Amir Arain, Giridhar P Kalamangalam, Suzette M LaRoche, Bonilha Leonardo, Maysaa Basha, Nabil J Azar, Ekrem Kutluay, Gabriel U Martz, Wolf J Bethany, Chad G Waters, Brian C Dean
Affiliations

Characteristics of EEG Interpreters Associated With Higher Interrater Agreement

Jonathan J Halford et al. J Clin Neurophysiol. 2017 Mar.

Abstract

Purpose: The goal of the project is to determine characteristics of academic neurophysiologist EEG interpreters (EEGers), which predict good interrater agreement (IRA) and to determine the number of EEGers needed to develop an ideal standardized testing and training data set for epileptiform transient (ET) detection algorithms.

Methods: A three-phase scoring method was used. In phase 1, 19 EEGers marked the location of ETs in two hundred 30-second segments of EEG from 200 different patients. In phase 2, EEG events marked by at least 2 EEGers were annotated by 18 EEGers on a 5-point scale to indicate whether they were ETs. In phase 3, a third opinion was obtained from EEGers on any inconsistencies between phase 1 and phase 2 scoring.

Results: The IRA for the 18 EEGers was only fair. A select group of the EEGers had good IRA and the other EEGers had low IRA. Board certification by the American Board of Clinical Neurophysiology was associated with better IRA performance but other board certifications, years of fellowship training, and years of practice were not. As the number of EEGers used for scoring is increased, the amount of change in the consensus opinion decreases steadily and is quite low as the group size approaches 10.

Conclusions: The IRA among EEGers varies considerably. The EEGers must be tested before use as scorers for ET annotation research projects. The American Board of Clinical Neurophysiology certification is associated with improved performance. The optimal size for a group of experts scoring ETs in EEG is probably in the 6 to 10 range.

PubMed Disclaimer

Conflict of interest statement

The authors have no conflicts of interest to disclose.

Figures

FIG. 1
FIG. 1
Interrater agreement (IRA) of the groups of EEG interpreters with best IRA for each group size.
FIG. 2
FIG. 2
Percentage change in consensus opinion as group size of best groups changes. For example, when group size changes from 2 to 3, the consensus opinion changes by approximately 7%.
FIG. 3
FIG. 3
Interrater agreement of best group and second best group for all group sizes.
FIG. 4
FIG. 4
Sensitivity and specificity of all EEG interpreters versus consensus opinion based on binary scoring.
See this image and copyright information in PMC

References

    1. Seidel S, Pablik E, Aull-Watschinger S, Seidl B, Pataraia E. Incidental epileptiform discharges in patients of a tertiary centre. Clin Neurophysiol. 2016;127:102–107. - PubMed
    1. van Donselaar CA, Schimsheimer RJ, Geerts AT, Declerck AC. Value of the electroencephalogram in adult patients with untreated idiopathic first seizures. Arch Neurol. 1992;49:231–237. - PubMed
    1. Fountain NB, Freeman JM. EEG is an essential clinical tool: pro and con. Epilepsia. 2006;47:23–25. - PubMed
    1. Halford JJ. Computerized epileptiform transient detection in scalp EEG: obstacles to progress and the example of computerized ECG interpretation. Clin Neurophysiol. 2009;120:1909–1915. - PubMed
    1. Benbadis SR. Errors in EEGs and the misdiagnosis of epilepsy: importance, causes, consequences, and proposed remedies. Epilepsy Behav. 2007;11:257–262. - PubMed