Assessment of inter-examiner agreement and variability in the manual classification of auditory brainstem response

Abstract

Background: The analysis of the Auditory Brainstem Response (ABR) is of fundamental importance to the investigation of the auditory system behaviour, though its interpretation has a subjective nature because of the manual process employed in its study and the clinical experience required for its analysis. When analysing the ABR, clinicians are often interested in the identification of ABR signal components referred to as Jewett waves. In particular, the detection and study of the time when these waves occur (i.e., the wave latency) is a practical tool for the diagnosis of disorders affecting the auditory system. Significant differences in inter-examiner results may lead to completely distinct clinical interpretations of the state of the auditory system. In this context, the aim of this research was to evaluate the inter-examiner agreement and variability in the manual classification of ABR.

Methods: A total of 160 ABR data samples were collected, for four different stimulus intensity (80dBHL, 60dBHL, 40dBHL and 20dBHL), from 10 normal-hearing subjects (5 men and 5 women, from 20 to 52 years). Four examiners with expertise in the manual classification of ABR components participated in the study. The Bland-Altman statistical method was employed for the assessment of inter-examiner agreement and variability. The mean, standard deviation and error for the bias, which is the difference between examiners' annotations, were estimated for each pair of examiners. Scatter plots and histograms were employed for data visualization and analysis.

Results: In most comparisons the differences between examiner's annotations were below 0.1 ms, which is clinically acceptable. In four cases, it was found a large error and standard deviation (>0.1 ms) that indicate the presence of outliers and thus, discrepancies between examiners.

Conclusions: Our results quantify the inter-examiner agreement and variability of the manual analysis of ABR data, and they also allows for the determination of different patterns of manual ABR analysis.

Figure 1

Latency values obtained for each Jewett wave as function of the intensity (dBHL). The shaded areas are bounded by the minimum and maximum values of latency found for each wave. The standard deviation, the central tendency and its 95% confidence interval are also presented.

Figure 2

Bland-Altman plot for annotation agreement between examiners 3 and 4, for wave I. The bias, its 95% confidence interval and mean are given.

Figure 3

Histogram of the bias for the examiners E3 and E4 for the wave I.

Figure 4

Bland-Altman plot for annotation agreement between examiners 3 and 4, for wave III. The bias, its confidence interval and mean are given.

Figure 5

Histogram of the bias for the examiners E3 and E4 for the wave III.