Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Jul 2:2020:4942121.
doi: 10.1155/2020/4942121. eCollection 2020.

CT-TEE Image Registration for Surgical Navigation of Congenital Heart Disease Based on a Cycle Adversarial Network

Yunfei Lu #  1 , Bing Li #  2 , Ningtao Liu  1 , Jia-Wei Chen  1 , Li Xiao  3 , Shuiping Gou  1 , Linlin Chen  1 , Meiping Huang  4 , Jian Zhuang  5
Affiliations

CT-TEE Image Registration for Surgical Navigation of Congenital Heart Disease Based on a Cycle Adversarial Network

Yunfei Lu et al. Comput Math Methods Med. .

Abstract

Transesophageal echocardiography (TEE) has become an essential tool in interventional cardiologist's daily toolbox which allows a continuous visualization of the movement of the visceral organ without trauma and the observation of the heartbeat in real time, due to the sensor's location at the esophagus directly behind the heart and it becomes useful for navigation during the surgery. However, TEE images provide very limited data on clear anatomically cardiac structures. Instead, computed tomography (CT) images can provide anatomical information of cardiac structures, which can be used as guidance to interpret TEE images. In this paper, we will focus on how to transfer the anatomical information from CT images to TEE images via registration, which is quite challenging but significant to physicians and clinicians due to the extreme morphological deformation and different appearance between CT and TEE images of the same person. In this paper, we proposed a learning-based method to register cardiac CT images to TEE images. In the proposed method, to reduce the deformation between two images, we introduce the Cycle Generative Adversarial Network (CycleGAN) into our method simulating TEE-like images from CT images to reduce their appearance gap. Then, we perform nongrid registration to align TEE-like images with TEE images. The experimental results on both children' and adults' CT and TEE images show that our proposed method outperforms other compared methods. It is quite noted that reducing the appearance gap between CT and TEE images can benefit physicians and clinicians to get the anatomical information of ROIs in TEE images during the cardiac surgical operation.

PubMed Disclaimer

Conflict of interest statement

The authors declare that they have no conflicts of interest.

Figures

Figure 1
Figure 1
The architecture of CT-TEE image generation.
Figure 2
Figure 2
An illustration of discriminator.
Figure 3
Figure 3
Comparisons between original images and generated images. (a) Original TEE image. (b) CT-like image. (c) Histograms of TEE image and CT-like image. (d) Original CT image. (e) TEE-like image. (f) Histograms of CT image and TEE-like image.
Figure 4
Figure 4
The whole framework of the proposed method.
Figure 5
Figure 5
The network structure of CycleGAN.
Figure 6
Figure 6
The TEE-like images generated from CT images.
Figure 7
Figure 7
The registration result. (a) Original CT image. (b) Original TEE image. Results of (c) the proposed method. (d) FLIRT. (e) Demons registration.
Figure 8
Figure 8
The registration of an adult subject. (a) Original CT image. (b) CT-like image. (c) Ground truth on CT domain. (d) Ground truth on TEE domain. (e) Demons registration on original image. (f) Powell registration on original image. (g) Demons registration on generated image. (h) Powell registration on generated image.
Figure 9
Figure 9
The registration of a teenager subject. (a) Original CT image. (b) CT-like image. (c) Ground truth on CT domain. (d) Ground truth on TEE domain. (e) Demons registration on original image. (f) Powell registration on original image. (g) Demons registration on generated image. (h) Powell registration on generated image.
See this image and copyright information in PMC

References

    1. Cao X., Gao Y., Yang J., Wu G., Shen D. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2016. Learning-based multimodal image registration for prostate cancer radiation therapy; pp. 1–9. - DOI - PMC - PubMed
    1. Cao X., Yang J., Zhang J., et al. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2017. Deformable image registration based on similarity-steered cnn regression; pp. 300–308. - DOI - PMC - PubMed
    1. Fan J., Cao X., Xue Z., Yap P.-T., Shen D. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2018. Adversarial similarity network for evaluating image alignment in deep learning based registration; pp. 739–746. - DOI - PMC - PubMed
    1. Vercauteren T., Pennec X., Perchant A., Ayache N. Diffeomorphic demons: efficient non-parametric image registration. NeuroImage. 2009;45(1):S61–S72. doi: 10.1016/j.neuroimage.2008.10.040. - DOI - PubMed
    1. Radford A., Metz L., Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks. 2015. http://arxiv.org/abs/1511.06434.

MeSH terms