Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Mar 31:9:792390.
doi: 10.3389/fmed.2022.792390. eCollection 2022.

PET and CT Image Fusion of Lung Cancer With Siamese Pyramid Fusion Network

Ning Xiao  1 , Wanting Yang  1 , Yan Qiang  1 , Juanjuan Zhao  1 , Rui Hao  2 , Jianhong Lian  3 , Shuo Li  4
Affiliations

PET and CT Image Fusion of Lung Cancer With Siamese Pyramid Fusion Network

Ning Xiao et al. Front Med (Lausanne). .

Abstract

Background: The fusion of PET metabolic images and CT anatomical images can simultaneously display the metabolic activity and anatomical position, which plays an indispensable role in the staging diagnosis and accurate positioning of lung cancer.

Methods: In order to improve the information of PET-CT fusion image, this article proposes a PET-CT fusion method via Siamese Pyramid Fusion Network (SPFN). In this method, feature pyramid transformation is introduced to the siamese convolution neural network to extract multi-scale information of the image. In the design of the objective function, this article considers the nature of image fusion problem, utilizes the image structure similarity as the objective function and introduces L1 regularization to improve the quality of the image.

Results: The effectiveness of the proposed method is verified by more than 700 pairs of PET-CT images and elaborate experimental design. The visual fidelity after fusion reaches 0.350, the information entropy reaches 0.076.

Conclusion: The quantitative and qualitative results proved that the proposed PET-CT fusion method has some advantages. In addition, the results show that PET-CT fusion image can improve the ability of staging diagnosis compared with single modal image.

Keywords: PET-CT fusion; image quality; pyramid transform; siamese neural network; structural similarity.

PubMed Disclaimer

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figures

Figure 1
Figure 1
The strategy of the proposed medical image fusion method. The source images first feed into encoder composed of CLCM to extract feature. Then two modality features are fused by cross correlation layer. Finally, the fused PET-CT image is reconstructed through the deconvolutional decoder.
Figure 2
Figure 2
The channel coupling module. The channel coupling module utilizes two different pooling operation and feed results to multi-layer perceptron. The output of multi-layer perceptron continue to forward to element-wise summation and sigmoid operation.
Figure 3
Figure 3
The spatial pyramid coupling module. The spatial pyramid coupling module utilizes spatial pyramid pooling to get multi-scale feature maps and concatenate them.
Figure 4
Figure 4
The qualitative comparison results of patient A. (A) CT; (B) PET; (C) AD; (D) GF; (E) PAPCNN; (F) MCFNET; (G) IFCNN; (H) OURS.
Figure 5
Figure 5
The qualitative comparison results of patient B. (A) CT; (B) PET; (C) AD; (D) GF; (E) PAPCNN; (F) MCFNET; (G) IFCNN; (H) OURS.
Figure 6
Figure 6
The detail of fusion results of patient A. (A) CT; (B) PET; (C) AD; (D) GF; (E) PAPCNN; (F) MCFNET; (G) IFCNN; (H) OURS.
Figure 7
Figure 7
The detail of fusion results of patient B. (A) CT; (B) PET; (C) AD; (D) GF; (E) PAPCNN; (F) MCFNET; (G) IFCNN; (H) OURS.
See this image and copyright information in PMC

References

    1. Hartman TE, Swensen SJ. CT screening for lung cancer. Iaslc Thorac Oncol. (2018) 40:52–8. 10.1016/B978-0-323-52357-8.00007-X - DOI
    1. Decazes P, Hinault P, Veresezan O, Thureau S, Gouel P, Vera P. Trimodality PET/CT/MRI and radiotherapy: a mini-review. Front Oncol. (2021) 10:3392. 10.3389/fonc.2020.614008 - DOI - PMC - PubMed
    1. Zhou T, Lu H, Hu F, Shi H, Qiu S, Wang H. A new robust adaptive fusion method for double-modality medical image PET/CT. BioMed Res Int. (2021) 2021:8824395. 10.1155/2021/8824395 - DOI
    1. Luo Y, Wang Y, Zu C, Zhan B, Wu X, Zhou J, et al. . 3D transformer-GAN for high-quality PET reconstruction. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Strasbourg: Springer; (2021). p. 276–85. 10.1007/978-3-030-87231-1_27 - DOI
    1. Wang J, Li X, Zhang Y, Zhang X. Adaptive decomposition method for multi-modal medical image fusion. IET Image Process. (2018) 12:1403–12. 10.1049/iet-ipr.2017.1067 - DOI

LinkOut - more resources