Deep learning-based MR-to-CT synthesis: The influence of varying gradient echo-based MR images as input channels

Abstract

Purpose: To study the influence of gradient echo-based contrasts as input channels to a 3D patch-based neural network trained for synthetic CT (sCT) generation in canine and human populations.

Methods: Magnetic resonance images and CT scans of human and canine pelvic regions were acquired and paired using nonrigid registration. Magnitude MR images and Dixon reconstructed water, fat, in-phase and opposed-phase images were obtained from a single T₁ -weighted multi-echo gradient-echo acquisition. From this set, 6 input configurations were defined, each containing 1 to 4 MR images regarded as input channels. For each configuration, a UNet-derived deep learning model was trained for synthetic CT generation. Reconstructed Hounsfield unit maps were evaluated with peak SNR, mean absolute error, and mean error. Dice similarity coefficient and surface distance maps assessed the geometric fidelity of bones. Repeatability was estimated by replicating the training up to 10 times.

Results: Seventeen canines and 23 human subjects were included in the study. Performance and repeatability of single-channel models were dependent on the TE-related water-fat interference with variations of up to 17% in mean absolute error, and variations of up to 28% specifically in bones. Repeatability, Dice similarity coefficient, and mean absolute error were statistically significantly better in multichannel models with mean absolute error ranging from 33 to 40 Hounsfield units in humans and from 35 to 47 Hounsfield units in canines.

Conclusion: Significant differences in performance and robustness of deep learning models for synthetic CT generation were observed depending on the input. In-phase images outperformed opposed-phase images, and Dixon reconstructed multichannel inputs outperformed single-channel inputs.

Keywords: MR contrasts; deep learning; gradient echo; synthetic CT.

Figure 1

Transverse slices of the canine and human data sets (paired in‐phase MR images and CT scans). As compared with the human data set, the canine data set showed substantial intersubject variability in morphology (e.g., shape, size). Abbreviation: HU, Hounsfield unit

Figure 2

Description of how MR images from the T₁‐weighted multiple gradient‐echo (T1w‐MGE) were combined to form all 6 input configurations. Images were concatenated along the channel dimension, resulting in single‐channel input configurations (almost in phase [aIP] and almost opposed phase [aOP]) or multichannel input configurations (acquired images [Dual], Dixon reconstructed images [IPOP], water–fat decomposition [WF], and Dixon)

Figure 3

Two‐dimensional histograms between MRI intensities and CT Hounsfield units for voxels corresponding to cortical bone (HU > 200). The correlation between bone voxel intensities in CT scans and MR images is stronger in in‐phase (IP) images (A) than in opposed phase (OP) images (B)

Figure 4

Variation of mean absolute error (MAE) per subject averaged across the repeated experiments ($\overline{{\text{MAE}}^{\mathrm{R}}}$) for each input configuration. The shading represents the SD of MAE_body for each subject across replicates. To focus on variations between input configurations and remove intersubject variability, measurements were zero‐centered per subject: The average MAE across replicates and input configurations (MAE_Avg) was subtracted from $\overline{{\text{MAE}}^{\mathrm{R}}}$. Therefore, negative values for the relative MAE indicate better performance. For completeness, MAE_Avg per subject is present in the bar plot, indicating the intersubject variability

Figure 5

Computed tomography and synthesized CTs (sCTs) generated per input configurations for 3 subjects. The Dual configuration was omitted, as it was very similar to IPOP. A, Coronal view of the right femoral head and acetabulum of a human patient (subject 1). B, Transversal view of the pelvic anatomy of a canine subject (subject 1). C, Sagittal view of the spine of a canine subject (subject 12). Yellow arrows indicate hypo‐intense bone regions; red arrows indicate hyperintense regions; and the orange encircled area indicates the bowels that do not contain any air pockets on the sCTs

Figure 6

Comparison of sCTs generated by the input configurations for canine (A) and human (B) subjects with a focus on bone. The Dual input configuration was omitted, as it was very similar to IPOP. A region defined by the red square was enlarged and its window level adapted to highlight bone structures. Errors maps show the absolute errors between the sCTs and the CT. For the human patient, the red arrow shows a sclerotic (hyperintense) region of the acetabulum, called the acetabular sourcil, which cannot be easily distinguished from the rest of the acetabulum in the aOP‐based sCT

Figure 7

Surface distance maps obtained for human (subject 8) (A) and canine (subject 13) (B) subjects. Meshes were obtained by thresholding the CT at 200 HU and removing unconnected components. The color map indicates the bilateral surface distance between the CT and a sCT obtained using a Dixon input configuration. The high errors in the human subject, especially in the left femur, originate from registration errors between the CT and the MR/sCT. The canine bone rendering shows the baculum, a thin penile bone present only in a small fraction of the data set that resulted in higher errors for male subjects