Contour-Driven Atlas-Based Segmentation

Abstract

We propose new methods for automatic segmentation of images based on an atlas of manually labeled scans and contours in the image. First, we introduce a Bayesian framework for creating initial label maps from manually annotated training images. Within this framework, we model various registration- and patch-based segmentation techniques by changing the deformation field prior. Second, we perform contour-driven regression on the created label maps to refine the segmentation. Image contours and image parcellations give rise to non-stationary kernel functions that model the relationship between image locations. Setting the kernel to the covariance function in a Gaussian process establishes a distribution over label maps supported by image structures. Maximum a posteriori estimation of the distribution over label maps conditioned on the outcome of the atlas-based segmentation yields the refined segmentation. We evaluate the segmentation in two clinical applications: the segmentation of parotid glands in head and neck CT scans and the segmentation of the left atrium in cardiac MR angiography images.

Fig. 1

Left: CT image with segmentation of left parotid (yellow: patch-based, red: manual). Right: Magnification of the blue patch (top) with manual segmentation (bottom). The four most similar patches in the repository vote for background (black at the center location), although the patch belongs to the left parotid. Image patches are intensity normalized for visualization.

Fig. 2

Example segmentations of MR angiography images of the left atrium of the heart (yellow: automatic, red: manual). The initial atlas-based segmentation is established with intensity-weighted label fusion. We compare the refinement that only uses contour information from the image and the refinement that combines contours from the image and those from the label map.

Fig. 3

Three different voting schemes for labeling the location x in the test image (top) using the training image (bottom) and image transformation ${\widehat{\varphi }}_i$. Left: Only the information in the training image at the single location ${\widehat{\varphi }}_i\left(x\right)$ is considered. Middle: Information in a local region $N_{{\widehat{\varphi }}_i\left(x\right)}$ centered at ${\widehat{\varphi }}_i\left(x\right)$ is included, with higher weighting towards the center. Right: A non-local approach that integrates information from the entire image grid Ω.

Fig. 4

Gaussian process segmentation of the left parotid gland. The initial label from the atlas-based segmentation (bottom left) only partially agrees with the manual segmentation (top left). We extract contours from the image (top center) and use them to construct the kernel function k, see Eq. (27). The kernel defines a distribution over label maps $L\sim \mathcal{G}\mathcal{P}(0,k)$ supported by the image. Samples drawn from the Gaussian process illustrate possible segmentations of the image (bottom center). The manual segmentation is overlaid for reference. The samples exhibit sharp boundaries necessary for segmentation, and the correlation of locations in the parotid gland. The mode of the posterior distribution results in the refined segmentation, overlaid on the intensity image (top right) and on the initial label map (bottom right).

Fig. 5

Overview of the atlas-based segmentation with region-based voting on superpixels for left atrium. The test image and the initial label map L_o serve as input. First, contours are extracted from both input images, yielding mPb and lPb. Both are combined for the calculation of the spectral contour sPb and global boundary gPb. The contour gives rise to a hierarchical parcellation of the image, represented with the ultrametric contour map (UCM). Thresholding the UCM at level ρ yields superpixels at a specific granularity. Gaussian process inference with the kernel in Eq. (28) yields superpixel-wise voting and the final segmentation.

Fig. 6

Dice volume overlap (top) and modified Hausdorff distance (bottom) for the segmentation of left parotid (left) and right parotid (right) glands. Initial labels are created with a random forest (RF) classifier and non-local means (NLM) for different patch sizes. The refinement of label maps is done with contours (gPb, mPb) and superpixels (UCM), where we also evaluate the integration of label map contours (label). Center line indicates median, the boxes extend to the 25^th and 75^th percentiles, and the whiskers reach to the most extreme values not considered outliers (crosses).

Fig. 7

Examples slices for the patch-based segmentation and contour-driven refinement of the left parotid gland. Automatic segmentation results are shown in yellow, manual delineations are shown in red. The initial label maps created with non-local means (NLM) and random forests (RF) are shown in the left column. The superpixel refinement with UCM and the contour refinement with mPb are shown in the second and third column, respectively.

Fig. 8

Mean Dice score of the refined segmentation with UCM label as a function of the superpixel threshold ρ. The initial label map was created with the RF classifier on 7 × 7 × 3 patches for the left parotid gland.

Fig. 9

Dice volume overlap and modified Hausdorff distance for the segmentation of left (top two graphs) and right (bottom two graphs) parotid glands. Initial label maps are created with atlas-based voting schemes (majority, intensity, patch, region intensity, and region patch voting). For the contour-driven refinement, we set gPb and mPb as contour estimates. We further compare to using a local noise model (local) and incorporating label contours (label). For the superpixel refinement, we use UCM with label contours. Center line indicates the median, the boxes extend to the 25^th and 75^th percentiles, and the whiskers reach to the most extreme values not considered outliers (crosses).

Fig. 10

Dice volume overlap (left) and modified Hausdorff distance (right) for the segmentation of the left atrium. Initial label maps are created with atlas-based voting schemes (majority, intensity, patch, region intensity, and region patch voting). The label maps are refined with the superpixels in UCM with the label map contours information (UCM label). Center line indicates median, the boxes extend to the 25^th and 75^th percentiles, and the whiskers reach to the most extreme values not considered outliers (crosses).

Fig. 11

Examples slices for the atlas-based segmentation and contour-driven refinement of the left atrium. Automatic segmentation results are shown in yellow, manual delineations are shown in red. The initial label maps were created with majority and intensity-weighted voting. The label maps for intensity-weighted voting are refined with the superpixels in UCM also considering label map contours (UCM label).