Reliability of a computer-aided manual procedure for segmenting optical coherence tomography scans

Abstract

Purpose: To assess the within- and between-operator agreement of a computer-aided manual segmentation procedure for frequency-domain optical coherence tomography scans.

Methods: Four individuals (segmenters) used a computer-aided manual procedure to mark the borders defining the layers analyzed in glaucoma studies. After training, they segmented two sets of scans, an Assessment Set and a Test Set. Each set had scans from 10 patients with glaucoma and 10 healthy controls. Based on an analysis of the Assessment Set, a set of guidelines was written. The Test Set was segmented twice with a ≥1 month separation. Various measures were used to compare test and retest (within-segmenter) variability and between-segmenter variability including concordance correlations between layer borders and the mean across scans (n = 20) of the mean of absolute differences between local border locations of individual scans, MEAN{mean( ΔLBL )}.

Results: Within-segmenter reliability was good. The mean concordance correlations values for an individual segmenter and a particular border ranged from 0.999 ± 0.000 to 0.978 ± 0.084. The MEAN{mean( ΔLBL )} values ranged from 1.6 to 4.7 μm depending on border and segmenter. Similarly, between-segmenter agreement was good. The mean concordance correlations values for an individual segmenter and a particular border ranged from 0.999 ± 0.001 to 0.992 ± 0.023. The MEAN{mean( ΔLBL )} values ranged from 1.9 to 4.0 μm depending on border and segmenter. The signed and unsigned average positions were considerably smaller than the MEAN{mean( ΔLBL )} values for both within- and between-segmenter comparisons. Measures of within-segmenter variability were only slightly larger than those of between-segmenter variability.

Conclusions: When human segmenters are trained, the within-and between-segmenter reliability of manual border segmentation is quite good. When expressed as a percentage of retinal layer thickness, the results suggest that manual segmentation provides a reliable measure of the thickness of layers typically measured in studies of glaucoma.

FIGURE 1

(A) Horizontal midline scan of a control subject showing the borders segmented and the layers identified for thickness measurements. The dashed and solid borders are for segmenter D’s test and retest segmentations, respectively. (B) The border location as a function of the distance from the center of the fovea for test (dashed) and retest (solid) segmentations. (C) The black curves are the difference between the dashed and solid curves in panel B. The other colored curves are for a similar analysis of the other 19 scans. (D) The layer thickness as a function of the distance from the center of the fovea for test (dashed) and retest (solid) segmentations.

FIGURE 2

(A) Horizontal midline scan of a control subject showing the borders segmented by segmenter D (dashed) and by the other three segmenters (solid). (B) The border location as a function of the distance from the center of the fovea for segmenter D (dashed) and the other three segmenters (solid). (C) The black curves are the difference between the dashed and the average of the three solid curves in panel B. The other colored curves are for a similar analysis of the other 19 scans. (D) The layer thickness as a function of the distance from the center of the fovea for segmenter D (dashed) and the other three segmenters (solid).

FIGURE 3

(A) Segmentation of the vitreous/RNFL (red) border in which the upper edge of the white pixels are approximately followed. (B) The recommended segmentation of the scan from panel A showing the recommended procedure in which the inclusion of black regions is avoided. (C) An illustration of a segmentation of the vitreous/RNFL (red) border that ignores the white region due to a light reflex. (D) An illustration of a segmentation of the vitreous/RNFL (red) border that ignores a vitreal separation. (E, F) The placement of the RNFL/RGC border (orange) border on the temporal side of the fovea differed among segmenters depending on the criterion used to define the RNFL. (G, H) If a border were missing in the fovea (G), segmenters were instructed to extrapolate the border seen outside the fovea to a point on the vitreous/RNFL border in the center of the fovea (H).