Deep learning super-resolution magnetic resonance spectroscopic imaging of brain metabolism and mutant isocitrate dehydrogenase glioma

Abstract

Background: Magnetic resonance spectroscopic imaging (MRSI) can be used in glioma patients to map the metabolic alterations associated with IDH1,2 mutations that are central criteria for glioma diagnosis. The aim of this study was to achieve super-resolution (SR) MRSI using deep learning to image tumor metabolism in patients with mutant IDH glioma.

Methods: We developed a deep learning method based on generative adversarial network (GAN) using Unet as generator network to upsample MRSI by a factor of 4. Neural networks were trained on simulated metabolic images from 75 glioma patients. The performance of deep neuronal networks was evaluated on MRSI data measured in 20 glioma patients and 10 healthy controls at 3T with a whole-brain 3D MRSI protocol optimized for detection of d-2-hydroxyglutarate (2HG). To further enhance structural details of metabolic maps we used prior information from high-resolution anatomical MR imaging. SR MRSI was compared to ground truth by Mann-Whitney U-test of peak signal-to-noise ratio (PSNR), structure similarity index measure (SSIM), feature-based similarity index measure (FSIM), and mean opinion score (MOS).

Results: Deep learning SR improved PSNR by 17%, SSIM by 5%, FSIM by 7%, and MOS by 30% compared to conventional interpolation methods. In mutant IDH glioma patients proposed method provided the highest resolution for 2HG maps to clearly delineate tumor margins and tumor heterogeneity.

Conclusions: Our results indicate that proposed deep learning methods are effective in enhancing spatial resolution of metabolite maps. Patient results suggest that this may have great clinical potential for image guided precision oncology therapy.

Keywords: D-2-hydroxyglutarate; deep learning; glioma; isocitrate dehydrogenase; magnetic resonance spectroscopic imaging; super-resolution.

Figure 1.

Flowchart diagram of deep learning super-resolution (SR) for magnetic resonance spectroscopic imaging (MRSI). There are 3 main blocks: (1) the low-resolution metabolic maps are first filtered using spectral quality criteria, inpainted and denoised, (2) the denoised maps are interpolated to produce the initial SR maps, and (3) the initialized SR is input in the deep neural networks (Unet or GAN) to obtain the final SR image. The last block can be run in 3 ways: (1) for DLmethod_1 only the initial MRSI and deep neural networks are used, (2) for DLmethod_2 the results of DLmethod_1 are further improved by subsequent feature nonlocal means (FNLM) with prior MRI, and (3) for DLmethod_3 the initial MRSI and prior MRI are both input in the deep neural networks. The architecture of the generator network (Unet) and discriminator network that are part of GAN are shown on the bottom left.

Figure 2.

Super-resolution magnetic resonance spectroscopic imaging (MRSI) without prior MRI (DLmethod_1) in simulated NAA maps in healthy subjects (SHS) and simulated d-2-hydroxyglutarate (2HG) maps in patients (SPT) with glioma. Results obtained by Unet and GAN are compared to conventional interpolation methods (bicubic and total variation). Examples from 3 simulated healthy subjects and 3 simulated patients are shown, from left to right: high-resolution (HR) ground truth MRSI (184 × 184), low-resolution (LR) MRSI (46 × 46), upsampled MRSI (184 × 184) obtained by bicubic, total variation (TV), Unet, and GAN.

Figure 3.

Super-resolution magnetic resonance spectroscopic imaging (MRSI) aided by prior MRI in simulated high-resolution NAA maps in healthy subjects (SHHS) data and simulated high-resolution d-2-hydroxyglutarate (2HG) maps in patients (SHPT) with glioma data. Results obtained by deep learning DLmethod_2 (Unet + FNLM↓, GAN + FNLM↓) and DLmethod_3 (↑Unet2inputs, GAN2inputs↓) are compared to DLmethod_1 (↑Unet, ↑GAN) and conventional (↑bicubic, ↑weighted TV w/wo FNLM↓) methods. Prior MRI is used to improve super-resolution MRSI, either by feature nonlocal means (FNLM) after neural networks, or as a second input (2 inputs) in the neural networks. Examples from 2 simulated healthy subjects and 2 simulated patients are shown. High-resolution (HR) ground truth MRSI (1.3 × 1.3 mm²), low-resolution (LR) MRSI (5.2 × 5.2 mm²), upsampled MRSI (1.3 × 1.3 mm²), and anatomical MRI (FLAIR and MPRAGE [MPRG] at 1 × 1 mm²). Up and down arrows by the names of the top of the figure indicate images in the upper or lower row, respectively, for a given subject.

Figure 4.

In vivo super-resolution magnetic resonance spectroscopic imaging (MRSI) measured in glioma patients (Pt). Original low-resolution HGG maps measured with the size 46 × 46 (5.2 × 5.2 mm²) were upsampled to 184 × 184 (1.3 × 1.3 mm²) with the corresponding methods from Figure 1. First, the low-resolution (LR) maps are filtered by spectral quality (SQ), inpainted for missing voxels (IPT), and denoised by nonlocal means denoising (NLMD). After denoising, MRSI is upsampled either by bicubic interpolation, weighted total variation, UNet, or GAN. Anatomical FLAIR images are used as prior to obtain super-resolution MRSI by feature nonlocal means (FNLM). Up and down arrows by the names of the top of the figure indicate images in the upper or lower row, respectively, for a given subject.