Unsupervised inter-frame motion correction for whole-body dynamic PET using convolutional long short-term memory in a convolutional neural network

Abstract

Subject motion in whole-body dynamic PET introduces inter-frame mismatch and seriously impacts parametric imaging. Traditional non-rigid registration methods are generally computationally intense and time-consuming. Deep learning approaches are promising in achieving high accuracy with fast speed, but have yet been investigated with consideration for tracer distribution changes or in the whole-body scope. In this work, we developed an unsupervised automatic deep learning-based framework to correct inter-frame body motion. The motion estimation network is a convolutional neural network with a combined convolutional long short-term memory layer, fully utilizing dynamic temporal features and spatial information. Our dataset contains 27 subjects each under a 90-min FDG whole-body dynamic PET scan. Evaluating performance in motion simulation studies and a 9-fold cross-validation on the human subject dataset, compared with both traditional and deep learning baselines, we demonstrated that the proposed network achieved the lowest motion prediction error, obtained superior performance in enhanced qualitative and quantitative spatial alignment between parametric K_i and V_b images, and significantly reduced parametric fitting error. We also showed the potential of the proposed motion correction method for impacting downstream analysis of the estimated parametric images, improving the ability to distinguish malignant from benign hypermetabolic regions of interest. Once trained, the motion estimation inference time of our proposed network was around 460 times faster than the conventional registration baseline, showing its potential to be easily applied in clinical settings.

Keywords: Convolutional network; Long-short term memory; Motion correction; Parametric imaging; Whole-body dynamic PET.

Fig. 1.

The overall workflow of the proposed multiple-frame motion correction framework. The input sequence is a dynamic frame series, where each moving frame is paired with the reference frame.

Fig. 2.

The proposed displacement estimation network B-convLSTM. (A) The network structure, a multiple-frame 3-D U-Net with a convolutional LSTM layer integrated at the bottleneck; (B) The structure of a single encoder level; (C) The structure of a single decoder level.

Fig. 3.

Sample absolute motion prediction error maps of the estimated motion fields for each motion correction method, with subject-wise whole-body mean absolute prediction errors (mean ± standard deviation) annotated below.

Fig. 4.

Sample overlaid Patlak $K_i$ (red) and $V_b$ (green) images showing inter-frame motion and correction impacts in brain (upper), heart (middle), and liver (bottom). The arrows are highlighting significant motion-related spatial mismatch and the improved alignment by motion compensation.

Fig. 5.

Sample voxel-wise Patlak NFE maps of skull and brain (upper) and heart (bottom). The arrows are highlighting regions with hotspots indicating high fitting error.

Fig. 6.

Sample hypermetabolic ROI $K_i$ images after each inter-frame motion correction method. The arrows are highlighting significant textures.

Fig. 7.

The sample overlaid $K_i$ (red) / $V_b$ (green) images of the sensitivity test, with the arrows highlighting significant spatial mismatch and motion correction effect. (A) λ = 0.1; (B) λ = 1; (C) λ = 10; (D) λ = 100.