Optimal multiple surface segmentation with shape and context priors

Abstract

Segmentation of multiple surfaces in medical images is a challenging problem, further complicated by the frequent presence of weak boundary evidence, large object deformations, and mutual influence between adjacent objects. This paper reports a novel approach to multi-object segmentation that incorporates both shape and context prior knowledge in a 3-D graph-theoretic framework to help overcome the stated challenges. We employ an arc-based graph representation to incorporate a wide spectrum of prior information through pair-wise energy terms. In particular, a shape-prior term is used to penalize local shape changes and a context-prior term is used to penalize local surface-distance changes from a model of the expected shape and surface distances, respectively. The globally optimal solution for multiple surfaces is obtained by computing a maximum flow in a low-order polynomial time. The proposed method was validated on intraretinal layer segmentation of optical coherence tomography images and demonstrated statistically significant improvement of segmentation accuracy compared to our earlier graph-search method that was not utilizing shape and context priors. The mean unsigned surface positioning errors obtained by the conventional graph-search approach (6.30 ±1.58 μ m) was improved to 5.14±0.99 μ m when employing our new method with shape and context priors.

Fig. 1

(a) Terrain-like surfaces S_i and S_j intersect each (x, y)-column exactly one time. (b) The shape representation ${\mathrm{\Delta }}_{pq}^i$ between neighboring columns p and q on surface S_i; and the context representation ${\delta }_p^{ij}$ between surfaces S_i and S_j on column p.

Fig. 2

Arc-weighted graph construction for the incorporation of the shape prior penalty on surface S_i between neighboring columns p and q. The intra-column arcs are shown in orange with +∞ weight. The hard shape constraint $\mid {\mathrm{\Delta }}_{pq}^i-{\overline{\mathrm{\Delta }}}_{pq}^i\mid \le L_{pq}^i$ is enforced by green arcs. Here we suppose ${\overline{\mathrm{\Delta }}}_{pq}^i=0$ and $L_{pq}^i=2$. The shape prior penalty is incorporated by arcs with dashed lines (brown, purple, yellow, and gray). Here we assume ${\overline{\mathrm{\Delta }}}_{pq}^i=0$, [f_s(0)]′ = 0 and f_s(0) = 0. Target surface S_i cuts arcs with weight [f_s(1)]″ (brown) and [f_s(0⁺)]″ = f_s(1) − f_s(0) (yellow). The total weight is equal to f_s(2).

Fig. 3

Arc-weighted graph construction for the incorporation of the context prior constraints between surface S_i and surface S_j on column p. Two subgraphs G_i (red) and G_j (blue) are constructed. The hard context constraint $\mid {\delta }_p^{ij}-{\overline{\delta }}_p^{ij}\mid \le H_p^{ij}$ is incorporated by green arcs. Here ${\overline{\delta }}_p^{ij}=1,H_p^{ij}=1$. The context-prior penalty is enforced by gray and purple arcs. We assume that [f_c(0)]′ = 0 and [f_c(0)] = 0. Target surface set S = {S_i, S_j} cuts arcs with weight [f_c(0⁺)]″ (gray). The total weight is equal to f_c(1).

Fig. 4

Intraretinal layers in 3-D OCT images. (a) A 2-D slice from the center of a volumetric OCT image—the dip corresponds to the fovea. (b) Seven surfaces (labeled 1–7) and corresponding retinal layers (NFL: nerve fiber layer; GCL+IPL: ganglion cell layer and inner plexiform layer; INL: inner nuclear layer; OPL: outer plexiform layer; ONL+IS: outer nuclear layer and photoreceptor inner segments; OS: photoreceptor outer segments and RPE: retinal pigment epithelium).

Fig. 5

Workflow for OCT intraretinal layer segmentation. The reported approach is utilized in the second step (indicated by the red box) to segment surfaces 2, 3, 4, 5 employing the shape and context priors.

Fig. 6

Visualization of the shape priors for surface 2 learned from the training set. The mean and the standard deviation are shown in the first and second rows, respectively. (a) x-direction. (b) y-direction.

Fig. 7

Learned surface context priorsin the form of the mean (the first row) and the standard deviation (the second row) between (a) surfaces (2,3); (b) surfaces (3,4); and (c) surfaces (4,5).

Fig. 8

Unsigned surface positioning errors observed in 28 volumetric OCT images. Inter-observer variability is shown in blue, errors of the original graph search method with only hard constraints are shown in red [8], and errors of the proposed method are shown in green. For all four surfaces, the resulting errors are significantly smaller than the corresponding inter-observer variabilities (p < 0.001). Compared with the original graph search method, our method showed a significant improvement for surfaces 2, 3, and 4 (p < 0.05).

Fig. 9

Intraretinal layer segmentation in 3-D OCT images. (a) A 2-D slice from 3-D retinal OCT dataset. (b) Seven manually labeled surfaces (1–7). (c) Segmentation achieved using the former graph searching approach with only hard constraints—surfaces 2, 3, 4, 5. (d) Segmentation achieved using the proposed algorithm with shape and context prior penalties.

Fig. 10

Improvements of segmentation contributed by (a) shape-prior penalties and (b) context prior penalties. The first row shows two slices of OCT images with manual segmentation. Segmentation results of two slices with both shape and context prior penalties are shown in the second row. The third row illustrates results (a) using context-prior penalties only (no shape-prior penalties) and (b) using shape-prior penalties only (no context-prior penalties).

Fig. 11

Segmentation of the prostate and bladder. (a) Triangular meshes for the bladder (yellow) and the prostate (blue) based on initial models. (b) Corresponding graph construction. An example 2-D slice is presented. p(v) represents the column with respect to the vertex v on the mesh. Dots represent nodes n_i ∈ G_i. Two sub-graphs G₁ and G₂ are constructed for the segmentation of the bladder and the prostate, respectively. Note that in the region of interaction (dashed box), a corresponding column in G₂ exists for each column in G₁ with the same position. The inter-surface arcs (purple) between corresponding columns enforce the surface context constraints in the interacting region.

Fig. 12

Simultaneous segmentation of the bladder (yellow) and the prostate (blue) in 3-D CT images using the graph-search approach with shape and context priors. (a), (b) Transverse views. (c) Sagittal view. (d) Coronal view. (e) 3-D representation of the segmentation result.