Estimating CT Image From MRI Data Using Structured Random Forest and Auto-Context Model

Abstract

Computed tomography (CT) imaging is an essential tool in various clinical diagnoses and radiotherapy treatment planning. Since CT image intensities are directly related to positron emission tomography (PET) attenuation coefficients, they are indispensable for attenuation correction (AC) of the PET images. However, due to the relatively high dose of radiation exposure in CT scan, it is advised to limit the acquisition of CT images. In addition, in the new PET and magnetic resonance (MR) imaging scanner, only MR images are available, which are unfortunately not directly applicable to AC. These issues greatly motivate the development of methods for reliable estimate of CT image from its corresponding MR image of the same subject. In this paper, we propose a learning-based method to tackle this challenging problem. Specifically, we first partition a given MR image into a set of patches. Then, for each patch, we use the structured random forest to directly predict a CT patch as a structured output, where a new ensemble model is also used to ensure the robust prediction. Image features are innovatively crafted to achieve multi-level sensitivity, with spatial information integrated through only rigid-body alignment to help avoiding the error-prone inter-subject deformable registration. Moreover, we use an auto-context model to iteratively refine the prediction. Finally, we combine all of the predicted CT patches to obtain the final prediction for the given MR image. We demonstrate the efficacy of our method on two datasets: human brain and prostate images. Experimental results show that our method can accurately predict CT images in various scenarios, even for the images undergoing large shape variation, and also outperforms two state-of-the-art methods.

Fig. 1

A pair of corresponding MR and CT images from the same human brain. Both air and bone have very low responses in MR image, but they are highly differentiable in CT image.

Fig. 2

Multi-scale feature extraction. At each location, the feature vector is comprised of the spatial coordinates of the location and also the features extracted from an image patch centered at that location with fixed size across all scales.

Fig. 3

Illustration of classic random forest and structured random forest. The top row shows an MR image patch, used to predict a corresponding CT value ν or a CT patch ν (shown in the bottom row). In the classic random forest, the input feature vector derived from MR image patch is used to predict a target value ν for a voxel (red point) in the CT image, while, in the structured random forest, the same input feature vector u is used to predict all values ν in a target CT patch (marked by the red rectangle).

Fig. 4

An example of the output results from 3 different decision trees in the random forest.

Fig. 5

Qualitative comparison of the prediction results from different variants of random forest for two datasets: 1) brain (upper panel) and 2) prostate (lower panel). In each panel, from left to right, the top row shows the prediction results using classic random forest with the mean ensemble model (C-M), structured random forest with the mean ensemble model (S-M), classic random forest with the median of means ensemble model (C-MM), structured random forest with the median of means ensemble model (S-MM), and the CT ground truth; The bottom row shows the close-ups of the regions indicated by red rectangles in the top row.

Fig. 6

Contribution of auto-context model. Top: results for the brain dataset. Bottom: results for the prostate dataset.

Fig. 7

Mean and standard deviation of MAE and PSNR measurements at different iterations of auto-context models in two datasets (left two: brain, right two: prostate). Both MAE and PSNR are measured in the entire image domain.

Fig. 8

Examples of the prediction results from two datasets: 1) brain (upper panel) and 2) prostate (lower panel). In each panel, from left to right, the top row shows the MR image, ground-truth CT image, and the predicted CT images given by our method, the sparse representation (SR) based method, and the atlas-based method, respectively; The bottom row shows the residual images by subtracting the ground-truth CT image from the prediction results given by our method, the sparse representation (SR) based method, and the atlas-based method, respectively.

Fig. 9

Tissue-wise voxel misclassification maps for two datasets. Red denotes false negative, while blue denotes false positive.

Fig. 10

Differential Dose-Volume Histograms (DVHs) of the prostate with real CT image (red) and our synthetic CT image (blue) for a typical patient.