Anatomically aided PET image reconstruction using deep neural networks

Abstract

Purpose: The developments of PET/CT and PET/MR scanners provide opportunities for improving PET image quality by using anatomical information. In this paper, we propose a novel co-learning three-dimensional (3D) convolutional neural network (CNN) to extract modality-specific features from PET/CT image pairs and integrate complementary features into an iterative reconstruction framework to improve PET image reconstruction.

Methods: We used a pretrained deep neural network to represent PET images. The network was trained using low-count PET and CT image pairs as inputs and high-count PET images as labels. This network was then incorporated into a constrained maximum likelihood framework to regularize PET image reconstruction. Two different network structures were investigated for the integration of anatomical information from CT images. One was a multichannel CNN, which treated PET and CT volumes as separate channels of the input. The other one was multibranch CNN, which implemented separate encoders for PET and CT images to extract latent features and fed the combined latent features into a decoder. Using computer-based Monte Carlo simulations and two real patient datasets, the proposed method has been compared with existing methods, including the maximum likelihood expectation maximization (MLEM) reconstruction, a kernel-based reconstruction and a CNN-based deep penalty method with and without anatomical guidance.

Results: Reconstructed images showed that the proposed constrained ML reconstruction approach produced higher quality images than the competing methods. The tumors in the lung region have higher contrast in the proposed constrained ML reconstruction than in the CNN-based deep penalty reconstruction. The image quality was further improved by incorporating the anatomical information. Moreover, the liver standard deviation was lower in the proposed approach than all the competing methods at a matched lesion contrast.

Conclusions: The supervised co-learning strategy can improve the performance of constrained maximum likelihood reconstruction. Compared with existing techniques, the proposed method produced a better lesion contrast versus background standard deviation trade-off curve, which can potentially improve lesion detection.

Keywords: anatomical prior; deep learning; image reconstruction; positron emission tomography.

Figure 1:

The architecture of three 3D deep neural networks with (a) PET input only, (b) PET/CT multi-channel input and (c) PET/CT multi-branch input, respectively. The input to each modality-specific encoder is a 3D patch of the corresponding modality.

Figure 2:

The training and validation losses as a function of epoch.

Figure 3:

CT images and corresponding cumulative histogram (a) before and (b) after histogram equalization.

Figure 4:

Reconstructed low-count images using different methods for the simulated test dataset. ADMM-MB-histequ denotes our proposed method method that uses the multi-branch input CNN constrained reconstruction with histogram equalized CT image. HC-MLEM represents MLEM reconstruction of high-count data for reference. Our proposed method is marked in bold face.

Figure 5:

(a) The contrast recovery (CR) versus background standard derivation (STD) curves for the simulated test dataset. (b) The MSE as a function of iteration number for the simulated test dataset. Markers are plotted for every 10 iterations. Deep learning based methods were initialized with the 30^th MLEM iteration.

Figure 6:

Reconstructed low-count images using different methods for the hybrid lesion test dataset. Our proposed method is marked in bold face. HC-MLEM denotes MLEM reconstruction of high-count data for reference.

Figure 7:

The contrast recovery (CR) versus background standard derivation (STD) curves for the hybrid lesion test dataset. Markers are plotted for every 10 iterations. Deep learning based methods were initialized with the 30^th MLEM iteration.

Figure 8:

Reconstructed low-count images using different methods for the lung cancer test dataset. Our proposed method is marked in bold face. HC-MLEM denotes MLEM reconstruction of high-count data for reference.

Figure 9:

The contrast recovery (CR) versus background standard derivation (STD) curves for the lung cancer test dataset. Markers are plotted for every 10 iterations. Deep learning based methods were initialized with the 30^th MLEM iteration.

Figure 10:

Reconstructed low-count images using different methods for the lung cancer validation dataset. Our proposed method is marked in bold face. HC-MLEM denotes MLEM reconstruction of high-count data for reference.