DeepHarmony: A deep learning approach to contrast harmonization across scanner changes

Abstract

Magnetic resonance imaging (MRI) is a flexible medical imaging modality that often lacks reproducibility between protocols and scanners. It has been shown that even when care is taken to standardize acquisitions, any changes in hardware, software, or protocol design can lead to differences in quantitative results. This greatly impacts the quantitative utility of MRI in multi-site or long-term studies, where consistency is often valued over image quality. We propose a method of contrast harmonization, called DeepHarmony, which uses a U-Net-based deep learning architecture to produce images with consistent contrast. To provide training data, a small overlap cohort (n = 8) was scanned using two different protocols. Images harmonized with DeepHarmony showed significant improvement in consistency of volume quantification between scanning protocols. A longitudinal MRI dataset of patients with multiple sclerosis was also used to evaluate the effect of a protocol change on atrophy calculations in a clinical research setting. The results show that atrophy calculations were substantially and significantly affected by protocol change, whereas such changes have a less significant effect and substantially reduced overall difference when using DeepHarmony. This establishes that DeepHarmony can be used with an overlap cohort to reduce inconsistencies in segmentation caused by changes in scanner protocol, allowing for modernization of hardware and protocol design in long-term studies without invalidating previously acquired data.

Keywords: Contrast harmonization; Deep learning; Magnetic resonance imaging.

Figure 1:

Preprocessed images from one subject from the overlap cohort depicting the four input (Protocol #1) and four target (Protocol #2) contrasts.

Figure 2:

Diagram of DeepHarmony U-Net Implementation. Convolutions with (2) or (1/2) indicate strided convolutions with stride of 2 or 1/2, respectively. C_I and C_O refer to the number of input and output contrasts, respectively. All convolutions are followed by a rectified linear unit (ReLU) and batch normalization (BN), except the final 1×1convolution, which does not use normalization.

Figure 3:

Validation graphs for deep network training. Dotted line represents chosen epoch for testing.

Figure 4:

Harmonized Protocol #1 T1-weighted images using REPLICA, MMBS, O2O, and DeepHarmony (DH). For comparison, the input contrast (Protocol #1) and the target contrast (Protocol #2) are displayed on the left side of the white, dashed line.

Figure 5:

Comparison of SSIM and MAE for T1-weighted, FLAIR, and T2-weighted contrasts over both whole head and brain masks. All pairwise comparisons are significant unless marked with “n.s.”.

Figure 6:

Representative sagittal slices for the same subject showing acquired images on the left and harmonized (using DeepHarmony) on the right.

Figure 7:

DSC and PVD between segmented volumes using data from Protocol #1 and Protocol #2. In the top plot, all pairwise comparisons are significant except when marked with “n.s.”. In the bottom plot, the only significant differences are between ACQ and the methods marked with a star.

Figure 8:

Longitudinal trajectories for cortical grey matter (in % from baseline). Protocol #1 is shown in blue and Protocol #2 is shown in orange.