SPARE: Sparse-view reconstruction challenge for 4D cone-beam CT from a 1-min scan

Abstract

Purpose: Currently, four-dimensional (4D) cone-beam computed tomography (CBCT) requires a 3-4 min full-fan scan to ensure usable image quality. Recent advancements in sparse-view 4D-CBCT reconstruction have opened the possibility to reduce scan time and dose. The aim of this study is to provide a common framework for systematically evaluating algorithms for 4D-CBCT reconstruction from a 1-min scan. Using this framework, the AAPM-sponsored SPARE Challenge was conducted in 2018 to identify and compare state-of-the-art algorithms.

Methods: A clinically realistic CBCT dataset was simulated using patient CT volumes from the 4D-Lung database. The selected patients had multiple 4D-CT sessions, where the first 4D-CT was used as the prior CT, and the rest were used as the ground truth volumes for simulating CBCT projections. A GPU-based Monte Carlo tool was used to simulate the primary, scatter, and quantum noise signals. A total of 32 CBCT scans of nine patients were generated. Additional qualitative analysis was performed on a clinical Varian and clinical Elekta dataset to validate the simulation study. Participants were blinded from the ground truth, and were given 3 months to apply their reconstruction algorithms to the projection data. The submitted reconstructions were analyzed in terms of root-mean-squared-error (RMSE) and structural similarity index (SSIM) with the ground truth within four different region-of-interests (ROI) - patient body, lungs, planning target volume (PTV), and bony anatomy. Geometric accuracy was quantified as the alignment error of the PTV.

Results: Twenty teams participated in the challenge, with five teams completing the challenge. Techniques involved in the five methods included iterative optimization, motion-compensation, and deformation of the prior 4D-CT. All five methods rendered significant reduction in noise and streaking artifacts when compared to the conventional Feldkamp-Davis-Kress (FDK) algorithm. The RMS of the three-dimensional (3D) target registration error of the five methods ranged from 1.79 to 3.00 mm. Qualitative observations from the Varian and Elekta datasets mostly concur with those from the simulation dataset. Each of the methods was found to have its own strengths and weaknesses. Overall, the MA-ROOSTER method, which utilizes a 4D-CT motion model for temporal regularization, had the best and most consistent image quality and accuracy.

Conclusion: The SPARE Challenge represents the first framework for systematically evaluating state-of-the-art algorithms for 4D-CBCT reconstruction from a 1-min scan. Results suggest the potential for reducing scan time and dose for 4D-CBCT. The challenge dataset and analysis framework are publicly available for benchmarking future reconstruction algorithms.

Keywords: 4D-CBCT; grand challenge; image reconstruction.

Figure 1

The flowchart of the sparse‐view reconstruction challenge. [Color figure can be viewed at wileyonlinelibrary.com]

Figure 2

The definitions of ${\text{ROI}}_{\mathrm{Body}}$, ${\text{ROI}}_{\mathrm{Lung}}$, ${\text{ROI}}_{\mathrm{PTV}}$, and ${\text{ROI}}_{\mathrm{Bony}}$ for the quantitative analysis. [Color figure can be viewed at wileyonlinelibrary.com]

Figure 3

The end‐inhale phase of the ground truth, conventional Feldkamp–Davis–Kress reconstruction, and reconstruction from the five methods for the overall best performing case in the simulated dataset, that is, the highest ${\text{SSIM}}_{\mathrm{Body}}$ values averaged over the five methods. The window level was adjusted in each panel to encompass the 0.5th and 99.5th percentile pixel intensities within ${\text{ROI}}_{\mathrm{Body}}$. The 4D animation for this case is included in the Supplementary Material as Data S1.

Figure 4

The end‐inhale phase of the ground truth, conventional FDK reconstruction, and reconstruction from the five methods for the overall worst performing case in the simulated dataset, that is, the lowest ${\text{SSIM}}_{\mathrm{Body}}$ values averaged over the five methods. The window level was adjusted in each panel to encompass the 0.5th and 99.5th percentile pixel intensities within ${\text{ROI}}_{\mathrm{Body}}$. The 4D animation for this case is included in the Supplementary Material as Data S2.

Figure 5

Boxplots of the root‐mean‐squared‐error (top row) and structural similarity index (bottom row) values in different region‐of‐interests for the conventional Feldkamp–Davis–Kress (as the baseline) and the five methods when applied to the simulated dataset. [Color figure can be viewed at wileyonlinelibrary.com]

Figure 6

${\text{RMSE}}_{\mathrm{Body}}$ (left) and ${\text{SSIM}}_{\mathrm{Body}}$ (right) of the conventional Feldkamp–Davis–Kress reconstruction and the five methods when applied to cases of different simulation types, that is, scatter‐free, normal, and low dose.

Figure 7

Translation error in planning target volume position for the five methods when applied to the simulated dataset presented as boxplots of the (a) three‐dimensional error magnitude and (b) the magnitude of systematic and random LR, SI, and AP error. The systematic and random error was calculated as the mean and standard deviation of the translation error over the ten phases of a four‐dimensional cone‐beam computed tomography reconstruction. [Color figure can be viewed at wileyonlinelibrary.com]

Figure 8

The end‐inhale phase of the reference reconstruction (fully sampled), conventional Feldkamp–Davis–Kress reconstruction (downsampled), and reconstruction from the five methods for an example case in the Varian dataset. The window level was adjusted in each panel to encompass the 0.5th and 99.5th percentile pixel intensities within ${\text{ROI}}_{\mathrm{Body}}$. The four‐dimensional animation for this case is included in the Supplementary Material as Data S3.

Figure 9

The end‐inhale phase of the reference reconstruction (fully sampled), conventional FDK reconstruction (downsampled), and reconstruction from the five methods for an example case in the Elekta dataset. The window level was adjusted in each panel to encompass the 0.5th and 99.5th percentile pixel intensities within ${\text{ROI}}_{\mathrm{Body}}$. The four‐dimensional animation for this case is included in the Supplementary Material as Data S4.