Real-time radial reconstruction with domain transform manifold learning for MRI-guided radiotherapy

Abstract

Background: MRI-guidance techniques that dynamically adapt radiation beams to follow tumor motion in real time will lead to more accurate cancer treatments and reduced collateral healthy tissue damage. The gold-standard for reconstruction of undersampled MR data is compressed sensing (CS) which is computationally slow and limits the rate that images can be available for real-time adaptation.

Purpose: Once trained, neural networks can be used to accurately reconstruct raw MRI data with minimal latency. Here, we test the suitability of deep-learning-based image reconstruction for real-time tracking applications on MRI-Linacs.

Methods: We use automated transform by manifold approximation (AUTOMAP), a generalized framework that maps raw MR signal to the target image domain, to rapidly reconstruct images from undersampled radial k-space data. The AUTOMAP neural network was trained to reconstruct images from a golden-angle radial acquisition, a benchmark for motion-sensitive imaging, on lung cancer patient data and generic images from ImageNet. Model training was subsequently augmented with motion-encoded k-space data derived from videos in the YouTube-8M dataset to encourage motion robust reconstruction.

Results: AUTOMAP models fine-tuned on retrospectively acquired lung cancer patient data reconstructed radial k-space with equivalent accuracy to CS but with much shorter processing times. Validation of motion-trained models with a virtual dynamic lung tumor phantom showed that the generalized motion properties learned from YouTube lead to improved target tracking accuracy.

Conclusion: AUTOMAP can achieve real-time, accurate reconstruction of radial data. These findings imply that neural-network-based reconstruction is potentially superior to alternative approaches for real-time image guidance applications.

Keywords: MRI; deep learning; radiotherapy.

FIGURE 1

Deep neural networks as a fast, accurate reconstruction technique for tumor tracking applications. (a) Workflow showing the potential role of AUTOMAP in a radiotherapy treatment with dynamic beam adaptation. Dynamic MRI scans are acquired on an MRI‐linac and reconstructed in real time with AUTOMAP. A template‐matching algorithm extracts the target position from images and dynamically adapts the X‐ray beam via a multi‐leaf collimator (MLC). (b) The deep neural network architecture implemented to reconstruct an $n\times n$ image from radially sampled MRI data with AUTOMAP. Radial k‐space data are flattened into a 1D vector to create the input to a series of dense and convolutional layers that reconstruct an image.

FIGURE 2

Simulating patient motion during radial acquisitions. The CoMBAT phantom inputs respiratory and ECG traces to simulate patient anatomy during cardiothoracic motion. MR slices are simulated at each timepoint during the acquisition (red shading) and encoded to a golden‐angle radial trajectory. A “motion‐encoded” k‐space is derived by taking individual spokes from the “static” k‐space at individual timepoints during the acquisition. The anatomy change between start and end timepoints is shown as a difference image.

FIGURE 3

Quality of radial AUTOMAP reconstruction in comparison to conventional techniques. Reconstruction quality as measured via structural similarity and normalized root mean square error metrics (Norm. RMSE) for different undersampling factors. In general, a more accurate image reconstruction technique yields a high structural similarity value and a low normalized RMSE value. Results are shown for AUTOMAP (red), compressed sensing (CS, blue), and nonuniform fast Fourier transform (NUFFT, black). Resulting image quality was assessed for clean radial input data (shown in a) and for the same data with 25 dB of additive white Gaussian noise (shown in b). Data markers have been offset from the acceleration factor values shown to aid visual clarity. Error bars represent the standard deviation of metrics across the test dataset. *p < 0.05, **p < 0.01, ***p < 0.001, ns = no significant difference

FIGURE 4

Visual comparison of AUTOMAP reconstruction performance to conventional techniques. AUTOMAP is compared to compressed sensing (CS) and nonuniform fast Fourier transform (NUFFT) techniques in images reconstructed from 4× undersampled golden‐angle radial data. Images from clean radial data (a) and from data with 25‐dB additive white Gaussian noise (b) are shown. Normalized root‐mean‐square error (NRMSE), structural similarity (SSIM), and peak signal‐to‐noise ratio (PSNR) metrics are shown. Yellow arrows in a indicate streaking artifacts (zooming on the electronic version may aid visibility). Red arrows in (b) are provided for discussion in the text.

FIGURE 5

Reconstructing motion‐corrupted, undersampled data. (a) Reconstruction quality as measured via structural similarity and normalized root mean square error metrics (Norm. RMSE) for static test images derived from the ImageNET database and for motion‐corrupted test inputs derived from the YouTube 8M database. Results are shown for 4× undersampled data reconstructed with compressed sensing (blue), an AUTOMAP model trained on static data (red) and an AUTOMAP model trained on motion‐encoded data. Bars and lines are the mean and standard error of the mean calculated across 1000 test inputs. (b) Images reconstructed from k‐space data simulated with the CoMBAT phantom for a patient under routine cardiothoracic motion. Results for data reconstructed with compressed sensing, an AUTOMAP model trained on static data, and an AUTOMAP model trained on motion‐encoded data are shown. Normalized root‐mean‐square error (NRMSE), structural similarity (SSIM), and peak signal‐to‐noise ratio (PSNR) image quality metrics are evaluated against the last frame of the image sequence for motion‐encoded data. Yellow arrows indicate errors associated with the position of the diaphragm in reconstructed images.

FIGURE 6

Target tracking accuracy. (a) Regions of interest (ROIs) for the diaphragm (blue) and tumor (red) are defined in a ground truth image. (b) Displacement of ROIs defined in a as predicted by a template matching algorithm for the ground truth image sequence (yellow) and image sequences reconstructed using compressed sensing (CS, blue), a conventional AUTOMAP model (red) and an AUTOMAP‐motion model (purple). Steps in displacement reflect the underlying image resolution. Root mean square error values are calculated for the difference between target position in reconstructed image sequences and the ground truth image sequence. The motion trace input to the virtual phantom is shown (green) with a vertical offset for visibility.

FIGURE 7

Multichannel reconstruction performance of nonuniform fast‐Fourier transform root‐sum‐of‐squares (NUFFT RSS), Compressed sensing (CS) and AUTOMAP techniques. (a) Sensitivity maps used for four‐channel data acquisition and reconstruction with a reduction factor (R) of 8. (b) Average normalized root‐mean‐square error (NRMSE) and structural similarity metrics for each technique as measured across the test set (error bars denote standard deviation). (c) Visual comparison of reconstruction performance. NRMSE, structural similarity (SSIM), and peak signal‐to‐noise ratio (PSNR) metrics are shown. Error bars represent the standard deviation of metrics across the test dataset. *p < 0.05, **p < 0.01, ***p < 0.001, ns = no significant difference