Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Jan;6(1):014001.
doi: 10.1117/1.JMI.6.1.014001. Epub 2019 Jan 10.

Multiorgan segmentation using distance-aware adversarial networks

Roger Trullo  1 , Caroline Petitjean  1 , Bernard Dubray  2 , Su Ruan  1
Affiliations

Multiorgan segmentation using distance-aware adversarial networks

Roger Trullo et al. J Med Imaging (Bellingham). 2019 Jan.

Abstract

Segmentation of organs at risk (OAR) in computed tomography (CT) is of vital importance in radiotherapy treatment. This task is time consuming and for some organs, it is very challenging due to low-intensity contrast in CT. We propose a framework to perform the automatic segmentation of multiple OAR: esophagus, heart, trachea, and aorta. Different from previous works using deep learning techniques, we make use of global localization information, based on an original distance map that yields not only the localization of each organ, but also the spatial relationship between them. Instead of segmenting directly the organs, we first generate the localization map by minimizing a reconstruction error within an adversarial framework. This map that includes localization information of all organs is then used to guide the segmentation task in a fully convolutional setting. Experimental results show encouraging performance on CT scans of 60 patients totaling 11,084 slices in comparison with other state-of-the-art methods.

Keywords: convolutional neural networks; deep learning; distance map; generative adversarial networks; medical images; multiorgan; segmentation.

PubMed Disclaimer

Figures

Fig. 1
Fig. 1
Typical CT scan with manual segmentation. Some organs like the esophagus are especially challenging to segment.
Fig. 2
Fig. 2
(a) distance map overlapped on the CT scan and (b) GT labels showing heart in beige, esophagus in green and aorta in blue.
Fig. 3
Fig. 3
Proposed framework for multiclass segmentation using distance maps and generative adversarial networks. The details of the generator and discriminator networks are shown in Figs. 4 and 5.
Fig. 4
Fig. 4
Details of the generator (G) networks. The difference between the two networks is the number of feature maps in the last layer. The numbers indicate the number of feature maps at each layer.
Fig. 5
Fig. 5
Details of the discriminator (D) network (PatchGAN setting). The numbers indicate the number of feature maps at each layer.
Fig. 6
Fig. 6
Distance maps (a) generated by FCN, (b) generated by FCN + PatchGAN, or (c) obtained from segmentation GT.
Fig. 7
Fig. 7
Segmentation results in red versus GT in green. First row shows the results obtained by the GAN-samples strategy, while second row shows results obtained using the FCN-samples. (a) Esophagus, (b) heart, (c) trachea, and (d) aorta.
Fig. 8
Fig. 8
Data from the AAPM challenge. First column shows a slice of the CT scan with the manual segmentation of the OARs overlapped. The second and third columns show a 3-D rendering of the organs.
Fig. 9
Fig. 9
Scores obtained in the AAPM challenge by the participating methods and by the methods proposed in this paper. Scores from challenge methods (plus symbol) were taken from Ref. : DL1-5 refers to deep learning-based methods, MA1-2 refers to multiatlas methods. Scores from the methods presented in this paper (diamond symbol) are FCN, Fdm for FCN-dmaps, and Gdm for PatchGAN-dmaps.
Fig. 10
Fig. 10
Dice scores obtained in the AAPM challenge by the participating methods (reported from Ref.  and by the PatchGAN-dmaps (Gdm) method proposed in this paper. Each scatter point corresponds to one method, but we omitted the name of the method for the sake of clarity. We only mention where our method stands compared to the others; in particular one can see its Dice scores are better for the esophagus and the spinal cord.
See this image and copyright information in PMC

References

    1. Wang L., et al. , “Links: learning-based multi-source integration framework for segmentation of infant brain images,” NeuroImage 108, 160–172 (2015).NEIMEF10.1016/j.neuroimage.2014.12.042 - DOI - PMC - PubMed
    1. Tu Z., Bai X., “Auto-context and its application to high-level vision tasks and 3D brain image segmentation,” IEEE Trans. Pattern Anal. Mach. Intell. 32(10), 1744–1757 (2010).ITPIDJ10.1109/TPAMI.2009.186 - DOI - PubMed
    1. Criminisi A., et al. , Regression Forests for Efficient Anatomy Detection and Localization in CT Studies, pp. 106–117, Springer, Berlin, Heidelberg: (2011).
    1. Krizhevsky A., Sutskever I., Hinton G. E., “ImageNet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems 25, Pereira F., et al., Eds., Curran Associates, Inc., pp. 1097–1105 (2012).
    1. Simonyan K., Zisserman A., “Very deep convolutional networks for large-scale image recognition,” CoRR arXiv:1409.1556 (2014).