Combining deep learning with anatomical analysis for segmentation of the portal vein for liver SBRT planning

Abstract

Automated segmentation of the portal vein (PV) for liver radiotherapy planning is a challenging task due to potentially low vasculature contrast, complex PV anatomy and image artifacts originated from fiducial markers and vasculature stents. In this paper, we propose a novel framework for automated segmentation of the PV from computed tomography (CT) images. We apply convolutional neural networks (CNNs) to learn the consistent appearance patterns of the PV using a training set of CT images with reference annotations and then enhance the PV in previously unseen CT images. Markov random fields (MRFs) were further used to smooth the results of the enhancement of the CNN enhancement and remove isolated mis-segmented regions. Finally, CNN-MRF-based enhancement was augmented with PV centerline detection that relied on PV anatomical properties such as tubularity and branch composition. The framework was validated on a clinical database with 72 CT images of patients scheduled for liver stereotactic body radiation therapy. The obtained accuracy of the segmentation was [Formula: see text] 0.83 and [Formula: see text] 1.08 mm in terms of the median Dice coefficient and mean symmetric surface distance, respectively, when segmentation is encompassed into the PV region of interest. The obtained results indicate that CNNs and anatomical analysis can be used for the accurate segmentation of the PV and potentially integrated into liver radiation therapy planning.

Figure 1

Examples of portal vein manual segmentation (solid line) from liver CT images shown in axial, sagittal and coronal cross-sections. A relatively low level of contrast agent makes automated portal vein segmentation a challenging task.

Figure 2

A schematic illustration of the convolutional neural network (CNN) architecture used for enhancement of portal vein (PV) from CT images. Individual CNNs are applied on each orthogonal cross-section of the CT image in order to generate three PV enhancement maps. The maps are then averaged to form the resulting PV enhancement.

Figure 3

An example of structures that are visually very similar to portal vein in (a) CT images and may be erroneously enhanced by convolutional neural networks. (b) The portal vein is outlined with solid line whereas neighboring similar structures with dashed lines.

Figure 4

A schematic illustration of finding the portal vein centerline. Having a surface point x, i.e. a point with high magnitude of portal vein enhancement, meets three points y₁, y₂ and y₃ that can potentially represent portal vein surface. Point y₂ is selected as the optimal due to the high magnitude of its gradient and congruential gradient orientation against gradient of x

Figure 5

A schematic illustration of portal vein (PV) centerline detection. The original image is first analyzed by convolutional neural networks for enhancement of PV. To remove wrongly enhanced structures with the appearance similar to PV, we compute the locations where gradients computed on enhancement map mainly intersect. As PV can be roughly approximated with a set of tubular structures, the gradients on its surface intersect in the centerline of the PV.

Figure 6

Results of portal vein segmentation from CT images given in terms of Dice and symmetric surface distance. The results are computed using axial, coronal, sagittal and combined CNNs and with or without a predefined region of interest. Manual regions of interest (ROIs) restrict the area where the portal vein is searched for. The ROIs serve to exclude the disagreement between manual and automated segmentations originated from vaguely defined borders of the PV region important for radiotherapy planning. The disagreement, which cannot be straightforwardly considered as an error, occurs when manual segmentation stops at a specific branch point, whereas CNN continues enhancing PV along the branch.

Figure 7

Examples of segmentation results for four cases from the database. The results are shown in comparison to the original CT images (first raw) using axial (second raw) and coronal (third raw) cross-sections with segmentations. The Dice coefficient for each case is given in the fourth raw. The overlap between manual and automatic segmentation is colored in green, and mis-segmentations are colored in red.

Figure 8

Examples of (top) CT images with (bottom) the corresponding manual segmentations of portal vein (blue) that are challenging to segment automatically. (a) Example of low level of contrast agent. (b) Example of strong image artifacts originated from fiducial markers. (c) Example of vasculature stent.