Approaching expert results using a hierarchical cerebellum parcellation protocol for multiple inexpert human raters

Abstract

Volumetric measurements obtained from image parcellation have been instrumental in uncovering structure-function relationships. However, anatomical study of the cerebellum is a challenging task. Because of its complex structure, expert human raters have been necessary for reliable and accurate segmentation and parcellation. Such delineations are time-consuming and prohibitively expensive for large studies. Therefore, we present a three-part cerebellar parcellation system that utilizes multiple inexpert human raters that can efficiently and expediently produce results nearly on par with those of experts. This system includes a hierarchical delineation protocol, a rapid verification and evaluation process, and statistical fusion of the inexpert rater parcellations. The quality of the raters' and fused parcellations was established by examining their Dice similarity coefficient, region of interest (ROI) volumes, and the intraclass correlation coefficient of region volume. The intra-rater ICC was found to be 0.93 at the finest level of parcellation.

Figure 1

Illustration of cerebellar anatomy and lobule grouping.

Figure 2

An illustration of the lobules VIIB through VIIIB being consistently (above) and inconsistently delineated (below). Fissures are indicated with red lines. The anatomical differences between the two subjects were a significant source of error. In particular, the lower subject had an “extra” fissure that caused confusion among the inexpert raters.

Figure 3

A schematic showing the steps of the hierarchical delineation protocol.

Figure 4

Example of an automatically generated montage of rater delineations used for rapid rater review. The first example shows examples of axial, coronal, and sagittal slices for one subject delineated twice by three raters. All raters were consistent with themselves and with each other. The second example shows a “misattribution” error, in which Rater 3 delineated the boundaries correctly but switched the labels for the left and right.

Figure 5

Box plots of Dice similarity coefficient with the expert rater for the human raters and label fusion by STAPLER. The upper and lower plots show overlaps for the lobar parcellation and lobule parcellations of Figure 1, respectively.

Figure 6

Box plots of signed volume difference with the expert rater for the human raters and label fusion by STAPLER. The upper and lower plots show comparisons for the lobar parcellation and lobule parcellations of Figure 1, respectively.

Figure 7

Examples of lobule X delineations. The inexpert rater delineations are labeled with Dice similarity coefficients with the expert delineation. Note that small absolute errors translate to large relative errors due to the small size of the structure.

Figure 8

Box plots of Dice similarity coefficient for SUIT, multi-atlas segmentation and the proposed manual method using inexpert raters. The rightmost set of boxes shows the of the average Dice similarity across all labels.