Adaptive weighted log subtraction based on neural networks for markerless tumor tracking using dual-energy fluoroscopy

Abstract

Purpose: To present a novel method, based on convolutional neural networks (CNN), to automate weighted log subtraction (WLS) for dual-energy (DE) fluoroscopy to be used in conjunction with markerless tumor tracking (MTT).

Methods: A CNN was developed to automate WLS (aWLS) of DE fluoroscopy to enhance soft tissue visibility. Briefly, this algorithm consists of two phases: training a CNN architecture to predict pixel-wise weighting factors followed by application of WLS subtraction to reduce anatomical noise. To train the CNN, a custom phantom was built consisting of aluminum (Al) and acrylic (PMMA) step wedges. Per-pixel ground truth (GT) weighting factors were calculated by minimizing the contrast of Al in the step wedge phantom to train the CNN. The pretrained model was then utilized to predict pixel-wise weighting factors for use in WLS. For comparison, the weighting factor was manually determined in each projection (mWLS). A thorax phantom with five simulated spherical targets (5-25 mm) embedded in a lung cavity, was utilized to assess aWLS performance. The phantom was imaged with fast-kV dual-energy (120 and 60 kVp) fluoroscopy using the on-board imager of a commercial linear accelerator. DE images were processed offline to produce soft tissue images using both WLS methods. MTT was compared using soft tissue images produced with both mWLS and aWLS techniques.

Results: Qualitative evaluation demonstrated that both methods achieved soft tissue images with similar quality. The use of aWLS increased the number of tracked frames by 1-5% compared to mWLS, with the largest increase observed for the smallest simulated tumors. The tracking errors for both methods produced agreement to within 0.1 mm.

Conclusions: A novel method to perform automated WLS for DE fluoroscopy was developed. Having similar soft tissue quality as well as bone suppression capability as mWLS, this method allows for real-time processing of DE images for MTT.

Keywords: convolutional neural networks; dual-energy imaging; fast-kV switching; markerless tumor tracking.

Figure 1.

Visualization of a 15 mm target overlapping with vertebrae as a function of the weighting factor. Starting at the top left, the weighting factors are 0.45, 0.5, 0.55, 0.6, 0.65 and 0.7 respectively. An optimal weighting factor is 0.55 shown on top row third column

Figure 1.

Visualization of a 15 mm target overlapping with vertebrae as a function of the weighting factor. Starting at the top left, the weighting factors are 0.45, 0.5, 0.55, 0.6, 0.65 and 0.7 respectively. An optimal weighting factor is 0.55 shown on top row third column

Figure 2.

Demonstration of the calibration setup. The top image shows the wedge phantom on the treatment table in front of the OBI detector of the linac. A projection image of the of Al+PMMA phantom is shown in the lower left, while an image of the PMMA wedge alone is shown in the lower right.

Figure 3.

Soft tissue weighting factor obtained using the wedge calibration phantom for each pixel. The edges of the phantom corresponding to air or PMMA are assigned average values. Each horizontal strip represents a different thickness of Al.

Figure 4.

Flowchart of the training phase. Transmission intensity images (left), normalized to air, are passed into CNN architecture (middle) to predict corresponding weighting factors (right).

Figure 5.

Flowchart of the automated WLS method. I_h and I_l represents measured transmitted intensities of high and low kVp settings, respectively (left images). A mapping of the soft tissue weighting factors predicted using the pre-trained network is shown in the middle. The image on the right represents the DE image created by combining high and low energy images with the weighing factors using Equation 3.

Figure 6.

Demonstration of weighting factors obtained using neural networks. Projections of the anthropomorphic phantom at 60 kVp (top) and corresponding pixel-wise weighting factors for two different views. Higher values of the weighting factor are observed for bone pixels. The shoulder and abdominal region has higher values due to photon starvation.

Figure 7.

Projections of the CIRS phantom using 120kVp (left column), mWLS (middle column) and aWLS (right column) at 270 gantry angle. No expressive differences are observed between mWLS and aWLS techniques.

Figure 8.

Projections of the anthropomorphic thorax phantom using 120 kVp (left column), mWLS (middle column) and aWLS (right column) imaging at 270 and 315 gantry angles. No visual differences are observed between the both WLS techniques. Note: this phantom does not have an embeded tumor in the lung.