PVR: Patch-to-Volume Reconstruction for Large Area Motion Correction of Fetal MRI

Abstract

In this paper, we present a novel method for the correction of motion artifacts that are present in fetal magnetic resonance imaging (MRI) scans of the whole uterus. Contrary to current slice-to-volume registration (SVR) methods, requiring an inflexible anatomical enclosure of a single investigated organ, the proposed patch-to-volume reconstruction (PVR) approach is able to reconstruct a large field of view of non-rigidly deforming structures. It relaxes rigid motion assumptions by introducing a specific amount of redundant information that is exploited with parallelized patchwise optimization, super-resolution, and automatic outlier rejection. We further describe and provide an efficient parallel implementation of PVR allowing its execution within reasonable time on commercially available graphics processing units, enabling its use in the clinical practice. We evaluate PVR's computational overhead compared with standard methods and observe improved reconstruction accuracy in the presence of affine motion artifacts compared with conventional SVR in synthetic experiments. Furthermore, we have evaluated our method qualitatively and quantitatively on real fetal MRI data subject to maternal breathing and sudden fetal movements. We evaluate peak-signal-to-noise ratio, structural similarity index, and cross correlation with respect to the originally acquired data and provide a method for visual inspection of reconstruction uncertainty. We further evaluate the distance error for selected anatomical landmarks in the fetal head, as well as calculating the mean and maximum displacements resulting from automatic non-rigid registration to a motion-free ground truth image. These experiments demonstrate a successful application of PVR motion compensation to the whole fetal body, uterus, and placenta.

Fig. 1.

Three view-planes for raw 3D data acquired through stacks of ssFSE images covering the whole uterus. The transverse (a) is the in-plane view, i.e., native 2D slice scan orientation. Motion causes streaky artifacts for multi-planar reconstructions (MPR) in orthogonal views (b) and (c) caused by both maternal and fetal movements between the acquisition of individual slices.

Fig. 2.

Illustration of the basic ideas behind reconstruction : A simplified example of a 2D HR grid sampling from a 2D LR grids (left) and a practical example of 3D fetal MRI using multiple overlapping stacks of slices, by reconstructing a 3D HR image with an isotropic voxel size from LR images with anisotropic voxel size.

Fig. 3.

Overview of the required modules of state-of-the-art SVR methods and main components introduced by previous work.

Fig. 4.

A schematic and modular overview of the proposed patch-to-volume reconstruction (PVR) framework. The key parts are 3D-3D registration, patch extraction, 2D-3D registration, super-resolution, and EM-based outlier removal. Core contributions of PVR are written in red and marked with asterisk.

Fig. 5.

An illustrative figure showing both square patches and superpixels methods for the patch extraction step. A 2D superpixel shows more flexibility than a square patch in extracting rigid regions or similar voxels. In practice, superpixels are dilated with few pixels to include some contextual information in order to increase the accuracy of the patch to volume registration step.

Fig. 6.

Example for the observed differences in the first iteration of a fetal brain MRI reconstruction (a). (b) shows a magnified region using a sinc function for the PSF similar to and (c) shows the result from using a Taylor series approximation of the sinc function as used in this work. Taylor series approximation allows a better approximating of small values close to zero. (d) shows the difference between both images.

Fig. 7.

The software modules defined for the implementation of the proposed approach. For implementation details, please refer to the provided source code.

Fig. 8.

Strong synthetic non-rigid motion artifacts caused by skewing an adult brain phantom with an angle of (). Rows: MRI in standard orientations: coronal, axial, and sagittal. Columns: original scan (), and sampled () axially, coronally and sagittally, and SVR, PVR-square patches and PVR-superpixel reconstructed isotropic images ().

Fig. 9.

Comparative reconstruction performance of SVR and PVR methods on synthetically corrupted Brainweb data using rigid translational transformations (1st column), bulk transformations (2nd column) and non-rigid skewing deformations (3rd column). Left to right: PSNR, SSIM and CC over skew angle in degrees and translations in mm for SVR (blue), superpixel-based PVR (, yellow) and PVR using square patches (, red).

Fig. 10.

Example reconstructions of consecutive MR scans of a moving fetus (kicking): input data (a) and corresponding cutting planes through an SVR- (b) and PVR-reconstructed (c) volumes. SVR produces blurry but readable results because of high data redundancy and outlier rejection through robust statistics. PVR with square patches of and appears visually superior. The arrow points at an area of substantial quality differences caused by independent rapid movements of the leg.

Fig. 11.

Three viewing planes through the original motion corrupted scan of a moving twin with a gestational age of 28 weeks (a) and PVR reconstruction using multi-scale superpixels (b). For this dataset we masked the uterus to save unnecessary computation time in areas containing maternal tissue. The white arrow points at a unilateral multicystic kidney of one of the twins.

Fig. 12.

A sample 2D cutting plane through a motion-corrupted fetal brain (a) and placenta (f), after PVR using square patches with and (b) and (g). The DSSIM heat map for a baseline before reconstruction (c) and (h), and after PVR (d) and (i). The average DSSIM of the fetal brain equals 0.497 (c) and 0.248 (d), while for the placenta equals to 0.491 (h) and 0.214 in (i).

Fig. 13.

(a) Number of generated patches and (b) necessary additional overhead pixels (%) of the different PVR variants versus their reconstruction PSNR quality of the whole uterus (see Table I-a). Optimal results are found in the upper left corner of the plots, i.e., high reconstruction quality and low computational overhead. The subject number is highlighted inside each circle marker. Multi-scale superpixels (MS-superpixel) achieve similar reconstruction quality to fixed-size (FS-patch), multi-scale (MS-patch) square patches while clustering in the area of minimal computational overhead.

Fig. 14.

Comparison of the best performing PVR parameters from for square patches and multi-scale superpixels with SVR for motion compensation of the brain in mature (GA > 33 weeks) fetuses with little motion. PVR’s image quality is similar to the small-area SVR method currently used in the clinical practice. The necessary patch overlap for PVR requires more computation, which results generally in longer runtime for any PVR variant.