Synthesizing Contrast-Enhanced MR Images from Noncontrast MR Images Using Deep Learning

Abstract

Background and purpose: Recent developments in deep learning methods offer a potential solution to the need for alternative imaging methods due to concerns about the toxicity of gadolinium-based contrast agents. The purpose of the study was to synthesize virtual gadolinium contrast-enhanced T1-weighted MR images from noncontrast multiparametric MR images in patients with primary brain tumors by using deep learning.

Materials and methods: We trained and validated a deep learning network by using MR images from 335 subjects in the Brain Tumor Segmentation Challenge 2019 training data set. A held out set of 125 subjects from the Brain Tumor Segmentation Challenge 2019 validation data set was used to test the generalization of the model. A residual inception DenseNet network, called T1c-ET, was developed and trained to simultaneously synthesize virtual contrast-enhanced T1-weighted (vT1c) images and segment the enhancing portions of the tumor. Three expert neuroradiologists independently scored the synthesized vT1c images by using a 3-point Likert scale, evaluating image quality and contrast enhancement against ground truth T1c images (1 = poor, 2 = good, 3 = excellent).

Results: The synthesized vT1c images achieved structural similarity index, peak signal-to-noise ratio, and normalized mean square error scores of 0.91, 64.35, and 0.03, respectively. There was moderate interobserver agreement between the 3 raters, regarding the algorithm's performance in predicting contrast enhancement, with a Fleiss kappa value of 0.61. Our model was able to accurately predict contrast enhancement in 88.8% of the cases (scores of 2 to 3 on the 3-point scale).

Conclusions: We developed a novel deep learning architecture to synthesize virtual postcontrast enhancement by using only conventional noncontrast brain MR images. Our results demonstrate the potential of deep learning methods to reduce the need for gadolinium contrast in the evaluation of primary brain tumors.

FIG 1.

Residual inception DenseNet (RID). A, RID model for virtual contrast enhancement (vT1c prediction) and enhancing tumor (ET) segmentation. B, RID model for whole tumor (WT) segmentation.

FIG 2.

Residual inception DenseNet (RID). A, RID model for whole tumor segmentation. B, RID model for virtual contrast enhancement and enhancing tumor segmentation.

FIG 3.

Building blocks of residual inception network. From left to right, dense block, convolution block, transition block, and projection block.

FIG 4.

Synthesized virtual contrast enhanced T1w (vT1c) images in 3 different subjects. Ground truth (left column) and synthesized vT1c (right column) image pairs for 9 subjects.

FIG 5.

Mosaic plot illustrating the distribution of 3 expert radiologists and their consensus along a 3-point Likert scale.

FIG 6.

Importance of input sequences example. Top row, input images: T1w, FLAIR, T2, and the ground truth T1c. Bottom row, output images with (A) all inputs (T1w, FLAIR, and T2w) given to the model, (B) T1w replaced with zeros in the input, (C) FLAIR replaced with zeros in the input, and (D) T2 replaced with zeros in the input. The T2 and FLAIR inputs together provide contrast enhancement prediction, whereas T1w input provides primarily anatomic detail.