Use of a deep learning neural network to generate bone suppressed images for markerless lung tumor tracking

Abstract

Background: Markerless tumor tracking (MTT) is being considered for real-time motion management of lung tumors. However, bony structures in conventional x-ray images can obfuscate the tumor, increasing tracking difficulty. Bone suppression using dual energy subtraction (DES) can improve tumor visibility but requires additional hardware or software that is not currently available with commercial on-board imaging (OBI) systems.

Purpose: This study compares DES images to synthetic DES (sDES) images generated by a U-net neural network, examining both image quality and tracking results.

Methods: High (120 kV) and low (60 kV) energy image pairs were obtained over 180-degree rotation using fast-kV switching for a motion phantom and 20 lung cancer patients. DES images were generated offline using weighted logarithmic subtraction. A U-net was then trained to transform 120 kV images into sDES images. Images from the phantom (2694 image pairs) and 20 patients (4499 image pairs), were divided into training, validation, and test sets consisting of 70%, 15%, and 15% of the images, and used for network training and evaluation. The similarity between sDES images and ground truth DES images were evaluated using histogram comparison, structural similarity index measure (SSIM), peak signal-to-noise ratio (PSNR), and the 2D correlation coefficient (2DCC). Separately, a template-tracking algorithm was used to predict tumor location on patient and phantom sDES images. Since there was no ground truth location for the patient images, the predicted locations of the tumor in the HE and sDES images were compared against the predicted locations in the DES images. For the phantom images, tracking success rate (TSR) was defined as the percentage of images in which the predicted and ground truth tumor location differed by <2 mm, missing frames (MF) was defined as the percentage of images in which the tracking algorithm failed, and the mean absolute error (MAE) was also calculated from the differences between predicted and ground truth locations of the tumor.

Results: Histogram count comparisons showed good agreement between the pixel distribution of sDES and DES images. Average SSIM, PSNR, and 2DCC scores for sDES images were 0.80 ± 0.05, 28.9 ± 3.4, and 0.97 ± 0.03 for phantom images, and 0.85 ± 0.04, 26.2 ± 3.5, and 0.97 ± 0.03 for patient images. For the patient images, the median tracking difference was 0.5 mm for HE versus DES images, and 0.3 mm for sDES versus DES images (p < 0.01). Separately, the TSR, MF, and MAE tracking metrics for the DES and sDES phantom images were found to be statistically equivalent, with scores of 93.5%, 0.22%, and 0.95 mm versus 93.5%, 0.22%, and 1.03 mm, respectively.

Conclusion: SDES images were found to be equivalent to DES images for use in MTT. The image similarity metrics comparing sDES and DES images showed good agreement, and MTT with DES and sDES images resulted in similar tracking metrics. These results indicate a trained U-net can be used to generate sDES images suitable for MTT using a single x-ray exposure without the need for additional hardware or software.

Keywords: deep learning; dual‐energy subtraction; intrafraction motion management.

FIGURE 1

A visual illustration of dual energy subtraction (DES) image creation, where a DES image is generated from the weighted logarithmic subtraction of a 60 kV low energy (LE) image from a 120 kV high energy (HE) image, using an empirically determined weighting factor w_s .

FIGURE 2

The schematic of the U‐net used in this work. The network consists of eight downsample blocks, followed by eight upsample blocks, with each block including a convolution layer, a batch normalization layer, and a (leaky) ReLU activation layer.

FIGURE 3

Comparison of high energy (HE), dual energy subtraction (DES), and synthetic DES (sDES) images generated by the U‐net, for both phantom and patient images. The red arrow in the image indicates the location of the tumor in the patient.

FIGURE 4

Comparison of high energy (HE), low energy (LE), dual energy subtraction (DES), and synthetic DES (sDES) images generated by the U‐net when using an LE image as input. The sDES images generated here are notably blurrier than in Figure 3.

FIGURE 5

Histogram comparison of pixel values for high energy (HE) in blue, dual energy subtraction (DES) in red, and synthetic DES (sDES) images in yellow (L 1), and purple (L2), for the phantom (top) and HE/LE patients (middle/bottom). The similarities in pixel value distribution between the DES and sDES images for the phantom (top) and HE patient (middle) images indicate the U‐net has learned HE to sDES image transformations mapping that are a good approximation to the DES image creation methodology described in Equation (1). In contrast, the network has difficulty mapping the LE patient (bottom) images to the DES pixel distribution.

FIGURE 6

Three plots showing a comparison between predicted tumor location and ground truth tumor location, with high energy (HE) tracking on top, dual energy subtraction (DES) tracking in the center, and synthetic DES (sDES) tracking on the bottom. Red dots indicate the predicted tumor location in each image, while the solid blue line indicates the programmed ground truth location. The tracking results shown here are from U‐net generated images using the L1‐norm as a loss function but are representative of the tracking results from all U‐net generated images.

FIGURE 7

Visual comparison of phantom tracking results between high energy (HE), dual energy subtraction (DES), and synthetic DES (sDES) images obtained from the tracking algorithm, obtained at 12 s (72°) intervals. The red arrows indicate the ground truth position of the spherical target, and the predicted location is outlined in green. The images shown here were generated from the U‐net using the L1‐norm loss function but are representative of the tracking results from all U‐net generated images.

FIGURE 8

Histogram differences in tracking prediction for high energy (HE) vs. dual energy subtraction (DES) patient images (top), and synthetic DES (sDES) vs. DES patient images (bottom). The smaller median difference in the sDES vs. DES images suggests that the sDES images provide an overall improvement in tracking compared to HE images.