Objective evaluation of reconstruction methods for quantitative SPECT imaging in the absence of ground truth

Abstract

Quantitative single-photon emission computed tomography (SPECT) imaging is emerging as an important tool in clinical studies and biomedical research. There is thus a need for optimization and evaluation of systems and algorithms that are being developed for quantitative SPECT imaging. An appropriate objective method to evaluate these systems is by comparing their performance in the end task that is required in quantitative SPECT imaging, such as estimating the mean activity concentration in a volume of interest (VOI) in a patient image. This objective evaluation can be performed if the true value of the estimated parameter is known, i.e. we have a gold standard. However, very rarely is this gold standard known in human studies. Thus, no-gold-standard techniques to optimize and evaluate systems and algorithms in the absence of gold standard are required. In this work, we developed a no-gold-standard technique to objectively evaluate reconstruction methods used in quantitative SPECT when the parameter to be estimated is the mean activity concentration in a VOI. We studied the performance of the technique with realistic simulated image data generated from an object database consisting of five phantom anatomies with all possible combinations of five sets of organ uptakes, where each anatomy consisted of eight different organ VOIs. Results indicate that the method provided accurate ranking of the reconstruction methods. We also demonstrated the application of consistency checks to test the no-gold-standard output.

Keywords: Evaluating reconstruction methods; No-gold-standard methods; Quantitative SPECT.

Figure 1

A schematic illustrating the problem statement for the no-gold-standard method

Figure 2

Various shapes taken by the beta distribution as the distribution parameters are varied

Figure 3

A demonstration of the effect of correcting the data for the bias and slope terms. In (a), we present an example plot of synthetically generated true and estimated mean activity concentration values. The estimated values are corrected for the slope and bias, and in (b), a plot of the true and corrected mean activity concentration values is presented. We observe that the noise is amplified due to the correction of the slope.

Figure 4

Regression lines estimated using the no-gold-standard method for the three reconstruction methods. The solid line is generated using the estimated linear model parameters, and the dashed line denotes the estimated standard deviation.