Statistical label fusion with hierarchical performance models

Abstract

Label fusion is a critical step in many image segmentation frameworks (e.g., multi-atlas segmentation) as it provides a mechanism for generalizing a collection of labeled examples into a single estimate of the underlying segmentation. In the multi-label case, typical label fusion algorithms treat all labels equally - fully neglecting the known, yet complex, anatomical relationships exhibited in the data. To address this problem, we propose a generalized statistical fusion framework using hierarchical models of rater performance. Building on the seminal work in statistical fusion, we reformulate the traditional rater performance model from a multi-tiered hierarchical perspective. This new approach provides a natural framework for leveraging known anatomical relationships and accurately modeling the types of errors that raters (or atlases) make within a hierarchically consistent formulation. Herein, we describe several contributions. First, we derive a theoretical advancement to the statistical fusion framework that enables the simultaneous estimation of multiple (hierarchical) performance models within the statistical fusion context. Second, we demonstrate that the proposed hierarchical formulation is highly amenable to the state-of-the-art advancements that have been made to the statistical fusion framework. Lastly, in an empirical whole-brain segmentation task we demonstrate substantial qualitative and significant quantitative improvement in overall segmentation accuracy.

Keywords: Hierarchical Segmentation; Label Fusion; Multi-Atlas Segmentation; Rater Performance Models; STAPLE.

Figure 1

Hierarchical representation of rater performance. A hierarchical model is developed for the brain, where, at each level, the performance of a rater is quantified. The overall quality of rater is then estimated through the unified hierarchical performance model.

Figure 2

Results on the motivating simulation. A simulated truth model was constructed to loosely model the types of relationships exhibited in the brain. The hierarchical formulations of STAPLE and Spatial STAPLE provide significant increases in overall segmentation accuracy. Here, the 4-level model results in statistically superior performance when compared to the 3- and 5-level models.

Figure 3

Quantitative results on the empirical whole-brain segmentation experiment. The hierarchical implementations of STAPLE, Spatial STAPLE, NLS, and NLSS provide statistically significant accuracy improvements across each of the considered label sets for the affine registration framework. Similarly, Hierarchical NLSS provides substantial accuracy improvements for the non-rigid registration framework.

Figure 4

Qualitative improvement exhibited by several state-of-the-art statistical fusion algorithms with the reformulated hierarchical performance model for the affine registration framework. For each of the considered statistical fusion algorithms we see substantial visual improvement for many of the considered labels. In particular, there appears to be marked improvement in the quality of the lateral ventricle labels and many of the cortical labels.