Inter- and intraobserver reliability and validation of a new method for determination of eosinophil counts in patients with esophageal eosinophilia

Abstract

Background: Diagnosis of eosinophilic esophagitis (EoE) requires quantification of esophageal eosinophilia.

Aims: The aims of this study were to assess inter- and intraobserver reliability for measuring esophageal eosinophil counts and to validate a novel method of determining tissue eosinophil density using digitized histopathology slides.

Methods: Patients were selected from the University of North Carolina EoE clinicopathologic database. Glass slides were de-identified and scanned to create digitized slides. Using a set protocol, 40 slides were read by each of three pathologists for interobserver measures, and were also reread by one pathologist as traditional glass slides. Different sets of 20 unique slides were read twice by each pathologist for intraobserver measures. Correlation and agreement were calculated with Pearson's rho and the kappa statistic.

Results: There was excellent correction between digitized images and glass slides (r = 0.91-0.95, P < 0.001). For maximum eosinophil densities, interobserver correlations were 0.91, 0.76, and 0.79. For mean densities, interobserver correlations were 0.90, 0.89, and 0.85. Intraobserver correlations for maximum densities were 0.99, 0.94, and 0.96, and for mean densities were 0.97, 0.87, and 0.89 (P < 0.001 for all correlations). Agreement was in the "substantial" to "near-perfect" range for pathologists using several diagnostic cut-points for EoE.

Conclusions: Both inter- and intraobserver correlations were excellent for determining eosinophil densities and counts. A method of using digitized slides was valid when compared with traditional glass slides. This protocol could be adopted for research and clinical purposes to further standardize the diagnostic process for EoE.

Fig. 1

View of a virtual esophageal biopsy slide using the Aperio ImageScope software. Note that the “Zoom” dialogue box (upper left portion of the screen) indicates a 5 × zoom, while the “Thumbnail” box (upper right portion of the screen) shows the location of the zoomed-in area in relation to the remainder of the biopsy specimen. (Permission for screenshot granted by Aperio Technologies.)

Fig. 2

View of Aperio software with an area of high eosinophil density selected with the “rectangle” tool. In this example, the “Annotations” box, seen in the lower right corner of the figure, displays the selected area as 140,193 µm². In this example, there are approximately 51 eosinophils within the selected area, for a eosinophil density of 364 eosinophils/mm², or an eosinophil count of 87 eos/hpf assuming a high-power field area of 0.24 mm², a typical size reported in the literature. (Permission for screenshot granted by Aperio Technologies.)

Fig. 3

Correlation between the digitized slides and traditional glass slides for determination of maximum eosinophil density (eos/mm²). a Digitized and microscope reads on the same slide set by pathologist 1. b Comparison of the microscope read with the average of all the maximum counts from all three pathologists

Fig. 4

Graphs of interobserver correlation for determination of maximum eosinophil density (eos/mm²) as calculated with Pearson’s rho. a Pathologist 1 versus pathologist 2. b Pathologist 1 versus pathologist 3. c Pathologist 2 versus pathologist 3

Fig. 5

Interobserver agreement measured with the κ statistic for selected diagnostic cut-points for EoE. The tissue count (eos/hpf) was back-calculated from the measured eosinophil density (eos/mm²) for an assumed microscope hpf of 0.24 mm²

Fig. 6

Intraobserver correlation for determination of maximum eosinophil density (eos/mm²) as calculated with Pearson’s rho. a Pathologist 1. b Pathologist 2. c Pathologist 3