Automated 3D reconstruction of the fetal thorax in the standard atlas space from motion-corrupted MRI stacks for 21-36 weeks GA range

Abstract

Slice-to-volume registration (SVR) methods allow reconstruction of high-resolution 3D images from multiple motion-corrupted stacks. SVR-based pipelines have been increasingly used for motion correction for T2-weighted structural fetal MRI since they allow more informed and detailed diagnosis of brain and body anomalies including congenital heart defects (Lloyd et al., 2019). Recently, fully automated rigid SVR reconstruction of the fetal brain in the atlas space was achieved in Salehi et al. (2019) that used convolutional neural networks (CNNs) for segmentation and pose estimation. However, these CNN-based methods have not yet been applied to the fetal trunk region. Meanwhile, the existing rigid and deformable SVR (DSVR) solutions (Uus et al., 2020) for the fetal trunk region are limited by the requirement of manual input as well the narrow capture range of the classical gradient descent based registration methods that cannot resolve severe fetal motion frequently occurring at the early gestational age (GA). Furthermore, in our experience, the conventional 2D slice-wise CNN-based brain masking solutions are reportedly prone to errors that require manual corrections when applied on a wide range of acquisition protocols or abnormal cases in clinical setting. In this work, we propose a fully automated pipeline for reconstruction of the fetal thorax region for 21-36 weeks GA range T2-weighted MRI datasets. It includes 3D CNN-based intra-uterine localisation of the fetal trunk and landmark-guided pose estimation steps that allow automated DSVR reconstruction in the standard radiological space irrespective of the fetal trunk position or the regional stack coverage. The additional step for generation of the common template space and rejection of outliers provides the means for automated exclusion of stacks affected by low image quality or extreme motion. The pipeline was quantitatively evaluated on a series of experiments including fetal MRI datasets and simulated rotation motion. Furthermore, we performed a qualitative assessment of the image reconstruction quality in terms of the definition of vascular structures on 100 early (median 23.14 weeks) and late (median 31.79 weeks) GA group MRI datasets covering 21 to 36 weeks GA range.

Keywords: Automated localisation; Automated pose estimation; Deformable slice-to-volume registration; Fetal MRI; Fetal heart.

Fig. 1. An example of a fetal CMR dataset (30 weeks GA).

A: Motion corrupted low resolution stacks acquired under different orientations visualised in the through plane view. B: The corresponding high-resolution SVR-reconstructed fetal thorax and 3D segmentation of the heart and vessels based on the pipeline proposed in Lloyd et al. (2019).

Fig. 2

An example of the global change of the fetal trunk (blue) and brain (red) position between stacks during acquisition for an early (23 weeks) GA case. This particular case was affected by severe motion with >90°rotations and this led to failed SVR reconstruction of the thorax.

Fig. 3

A. Comparison of the degree of the global fetal mobility during MRI acquisition for 55 randomly selected datasets acquired at St. Thomas’s Hospital and Evelina London Children’s Hospital using the same acquisition protocol: average rotation ranges for the fetal thorax ROI between stacks within individual datasets. It includes < 25 weeks GA early (red), 25–29 weeks GA (green), > 29 weeks GA late (blue) groups. B. Examples of fetal MRI scans at 22 and 32 weeks GA.

Fig. 4. Proposed pipeline for automated DSVR reconstruction of the fetal thorax from motion-corrupted MRI stacks.

Fig. 5. Multi-label 3D UNet network for 3D localisation of the fetal brain (red), fetal trunk (blue) and uterus (lilac) in motion-corrupted 3D MRI stacks.

Fig. 6. Proposed automated pipeline, Step I: 3D localisation of the fetal trunk in motion-corrupted stacks.

Fig. 7. Proposed automated pipeline, Step II: landmark-based 3D fetal thorax pose estimation and reorientation to the atlas space.

Fig. 8. An example of landmark-based reorientation to the atlas reference space

(A) an original motion-corrupted stack in random orientation with detected ROI-specific landmarks, (B) the corresponding 3D models in the original orientation and after transformation to the standard space based on point-based registration (pose estimation) and (C) the final reoriented stack.

Fig. 9. Proposed automated pipeline, Step III: selection of stacks, rejection of outliers and template generation.

Fig. 10. Proposed automated pipeline, Step IV: 3D DSVR reconstruction of the thorax ROI.

Fig. 11

An example of localisation results using multiple- (3) and single-label 3D UNet in three stacks with different ROI coverage: the whole uterus (A), trunk only (B), brain only (C). The segmentation outputs are visualised as blue (trunk) and red (brain) overlays.

Fig. 12

The results of organ localisation for 50 stacks from early and late GA cohorts (A-D). An example of reference (ground truth based on atlas label propagation) vs. network output for one of the early GA stacks is shown in (E).

Fig. 13

An example of simulated 30, 90 and 180 degrees rotations along one axis applied to a stack cropped to the fetal trunk ROI. The displayed landmark labels include: thorax (blue), heart (red), liver (brown) and abdomen (yellow).

Fig. 14

Simulated [0; 180] degrees range rotation experiment for comparison of the capture range of the classical rigid registration (red), automated landmark-based solution (blue) and combination of the classical registration initialised with the automated output (yellow): average global 3D NCC between the template and all transformed registered stacks in the masked thorax ROI.

Fig. 15. An example of reconstruction results for an early GA (23 weeks) dataset with 6 stacks affected by >90 degrees rotation motion

(A) original manual rigid SVR pipeline (Lloyd et al., 2019), (B) Steps I+IV, (C) Steps I+II+IV, (D) full pipeline with Steps I+II+III+IV. Note that all images were additionally aligned to the same space for visualisation purposes (axial and coronal views). The global change of the fetal thorax (blue) position between the different input stacks in this dataset is shown in (E).

Fig. 16

Fetal thorax reconstruction quality grading scheme for the heart ROI based on the proposed fully automated DSVR pipeline along with the examples from the early GA cohort.

Fig. 17

The results of qualitative assessment of fetal thorax reconstruction using the proposed automated DSVR pipeline: distribution of image quality scores for 50 early (B) 50 late GA (C) fetal MRI datasets. The GA distribution of the investigated datasets is presented in (A).

Fig. 18

The results of qualitative assessment of fetal thorax reconstruction using the original manual DSVR for the early GA cohort (50 cases): distribution of image quality scores (A) and an example of manual vs. automated reconstruction with different grades (B).

Fig. 19

An example of the proposed automated DSVR pipeline vs. the classical manual rigid SVR reconstruction (used for clinical studies inLloyd et al. (2019)) for severe (A) and minor (B) rotation and translation motion early GA (23 weeks) datasets.