A modality-adaptive method for segmenting brain tumors and organs-at-risk in radiation therapy planning

Abstract

In this paper we present a method for simultaneously segmenting brain tumors and an extensive set of organs-at-risk for radiation therapy planning of glioblastomas. The method combines a contrast-adaptive generative model for whole-brain segmentation with a new spatial regularization model of tumor shape using convolutional restricted Boltzmann machines. We demonstrate experimentally that the method is able to adapt to image acquisitions that differ substantially from any available training data, ensuring its applicability across treatment sites; that its tumor segmentation accuracy is comparable to that of the current state of the art; and that it captures most organs-at-risk sufficiently well for radiation therapy planning purposes. The proposed method may be a valuable step towards automating the delineation of brain tumors and organs-at-risk in glioblastoma patients undergoing radiation therapy.

Keywords: Generative probabilistic model; Glioma; Restricted Boltzmann machine; Whole-brain segmentation.

Figure 1:

Graphical representation of the model. The atlas-based prior on l is defined by parameters η governing the deformation of the atlas. The tumor-affected map z and the tumor core map y are connected to auxiliary variables H^z and H^y, respectively. The variables l, z and y jointly predict the data D according to the likelihood parameters θ. Shading indicates observed variables.

Figure 2:

A small 1D example of a cRBM with v = (v₁,…,v₇) and ${\mathbf{H}}^v={\left\{{\mathbf{h}}_m^v\right\}}_{m=1}^3.$ Visible units (image voxels) are connected to hidden units in a hidden group ${\mathbf{h}}_m^v$ through a convolutional filter ${\mathbf{w}}_m^v$ of size 3. All locations in v share the same filter weights. The connections are exemplified by the three central visible units which are connected to the central hidden unit in each group.

Figure 3:

The built atlas in axial, sagittal, and coronal view; shown in atlas space. Nodes and connections between nodes are shown in light green and probabilities of normal labels, interpolated between the nodes, are shown in varying colors (yellow = eye fluid, orange = eye tissue, red = optic nerves, green = brainstem, lilac = hippocampi, shades of blue = other normal labels).

Figure 4:

Segmentations of four representative subjects in the Copenhagen dataset. For each subject, the top row shows slices of the data (from left to right: T1c, CT, FLAIR, FLAIR² and T2), whereas the bottom row shows, from left to right, the manual segmentation and automatic segmentations for data combinations {T1c, FLAIR, T2, CT}, {T1c, FLAIR, T2} and {T1c, FLAIR²}. Label colors: white = TC, lilac = edema, green = BS, dark orange = HC, yellow/light orange = EB, red = ON/CH, shades of blue = other normal labels. For TC in order of appearance: Dice score: {0.68,0.67,0.62}, {0.93,0.93,0.91}, {0.86,0.85,0.85}, {0.61,0.72,0.73}, Hausdorff distance: {10,10,10}, {2,3, 5}, {7,7,6}, {42,25,8}.

Figure 5:

Boxplots of Dice scores (left) and Hausdorff distances (right) for structures in the Copenhagen dataset, for three data combinations in blue, red and green, respectively. 70 subjects in total. On each box, the central line is the median, the circle is the mean and the edges of the box are the 25th and 75th percentiles. Outliers are shown as dots. Black dots at the bottom of the Hausdorff distance boxplot indicate structures for which scores could not be calculated due to missing ground truth. Note that scores for the left and right structures are included separately in the box plots for HC, EB and ON.

Figure 6:

Hippocampi on two representative subjects in the Copenhagen dataset. Automatic segmentations (for {T1c, CT, FLAIR, T2}) in red and manual segmentations in green. Slice of segmentation overlaid on the T1-weighted scan and 3D surface plot of full structure. For left and right hippocampus: Dice score: {0.54,0.58},{0.63,0.67}; Hausdorff distance: {13,10}, {8,7}.

Figure 7:

Optic system on two representative subjects in the Copenhagen dataset. Automatic segmentations (for {T1c, CT, FLAIR, T2}) in red and manual segmentations in green. Slice of segmentation overlaid on the CT scan and 3D surface plot of full structure. For right and left eye; right and left optic nerve; and chiasm: Dice score: {0.91,0.89}, {0.91,0.87}, {0.67,0.67}, {0.48,0.55} and {0.49,0.44}; Hausdorff distance: {2,2}, {2,2}, {4,4}, {4,6} and {4,6}.

Figure 8:

Two problematic tumor core segmentations in the Copenhagen dataset. Data slices shown together with automatic segmentation (for {T1c, FLAIR, T2}) and manual segmentation. For tumor core: Dice score: {0.04,0.45}, Hausdorff distance: {41,28}.

Figure 9:

A radiation dose plan overlaid on a T1c image slice for a representative subject. The dose is measured in Gy.

Figure 10:

Dose volume histogram (DVH) of several structures for the representative subject in Figure 9, i.e., tumor core (GTV), brainstem (BS), hippocampi (HC), eyes (EB), optic nerves (ON), and chiasm (CH). Solid lines and broken lines correspond to automatic and manual segmentations, respectively. Note that all DVHs were computed using the original treatment dose plan, which was based on the manual segmentations.

Figure 11:

Summary statistics of DVH results for all subjects and structures, showing 5% volume (D5), 50% volume (D50), and 95% volume (D95), for manual versus automatic segmentations. Note that left and right hippocampus, eye and optic nerve are included as separate points in their respective plots.

Figure 12:

Three representative segmentations in the BRATS test dataset. Slices of T1c, FLAIR, T2, T1, and automatic segmentation. Label colors: white = TC, lilac = edema, green = BS, dark orange = HC, shades of blue = other brain tissues. Note that the images are skull-stripped by the BRATS challenge organizers.

Figure 13:

Box plots of Dice scores and Hausdorff distances for tumor core on the BRATS 2015 test dataset. 53 subjects in total. Scores are as reported in the challenge. On each box, the central line is the median, the circle is the mean and the edges of the box are the 25th and 75th percentiles. Outliers are shown as dots.

Figure 14:

Three representative segmentations in the London dataset. Slices of DIR, T2 and automatic segmentation.