Dual-core steered non-rigid registration for multi-modal images via bi-directional image synthesis

Abstract

In prostate cancer radiotherapy, computed tomography (CT) is widely used for dose planning purposes. However, because CT has low soft tissue contrast, it makes manual contouring difficult for major pelvic organs. In contrast, magnetic resonance imaging (MRI) provides high soft tissue contrast, which makes it ideal for accurate manual contouring. Therefore, the contouring accuracy on CT can be significantly improved if the contours in MRI can be mapped to CT domain by registering MRI with CT of the same subject, which would eventually lead to high treatment efficacy. In this paper, we propose a bi-directional image synthesis based approach for MRI-to-CT pelvic image registration. First, we use patch-wise random forest with auto-context model to learn the appearance mapping from CT to MRI domain, and then vice versa. Consequently, we can synthesize a pseudo-MRI whose anatomical structures are exactly same with CT but with MRI-like appearance, and a pseudo-CT as well. Then, our MRI-to-CT registration can be steered in a dual manner, by simultaneously estimating two deformation pathways: 1) one from the pseudo-CT to the actual CT and 2) another from actual MRI to the pseudo-MRI. Next, a dual-core deformation fusion framework is developed to iteratively and effectively combine these two registration pathways by using complementary information from both modalities. Experiments on a dataset with real pelvic CT and MRI have shown improved registration performance of the proposed method by comparing it to the conventional registration methods, thus indicating its high potential of translation to the routine radiation therapy.

Keywords: Image synthesis; Multi-modality; Non-rigid registration; Radiation therapy.

Fig. 1

Pelvic CT and MRI from the same patient, which scanned at different time points. Left two images: CT and labeled CT; Right two images: labeled MRI and MRI. The CT and MRI have already been linearly registered.

Fig. 2

The framework of the proposed dual-core steered MRI-to-CT image registration based on bi-directional image synthesis. The actual CT and actual MRI are already linearly aligned.

Fig. 3

An illustration of training and testing in patch-wise random forest for MR image synthesis from CT.

Fig. 4

An illustration of Haar-like features extraction within a single patch. Two ways of Haar-like feature generation: (a) the averaged intensities within a sub-block; (b) the averaged intensity differences between two sub-blocks.

Fig. 5

Iterative refinement of synthesized MRI with auto-context model.

Fig. 6

Visualization of the synthesized CT image (S-CT) from MRI for one subject.

Fig. 7

Visualization of the synthesized MR image (S-MRI) from CT for one subject.

Fig. 8

MRI-to-CT non-rigid registration results by using the proposed registration method. (a) The mean DSC values of prostate, bladder and rectum by different number of ACM layers in image synthesis. (b) The mean DSC values of prostate, bladder and rectum with respect to different DDF iterations in Algorithm 2. Note that, 3-iter DDF is applied in (a), while 2-layer ACM is used in (b).

Fig. 9

Examples of the synthesized images and the registration results (with ANTs-SyN) for 3 subjects in the dataset. Yellow contours are the contours of the prostate, bladder and rectum of the original CT image, used as ground-truth. Red Contours are the warped contours of the prostate, bladder and rectum after respective registrations. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 9

Examples of the synthesized images and the registration results (with ANTs-SyN) for 3 subjects in the dataset. Yellow contours are the contours of the prostate, bladder and rectum of the original CT image, used as ground-truth. Red Contours are the warped contours of the prostate, bladder and rectum after respective registrations. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)