A statistical modeling approach for evaluating auto-segmentation methods for image-guided radiotherapy

Abstract

We proposed a statistical modeling method for the quantitative evaluation of segmentation methods used in image guided radiotherapy. A statistical model parameterized on a Beta distribution was built upon the observations of the volume overlap between the segmented structure and the referenced structure. A statistical performance profile (SPP) was then estimated from the model using the generalized maximum likelihood approach. The SPP defines the probability density function characterizing the distribution of performance values and provides a graphical visualization of the segmentation performance. Different segmentation approaches may be influenced by image quality or observer variability. Our statistical model was able to quantify the impact of these variations and displays the underlying statistical performance of the segmentation algorithm. We demonstrated the efficacy of this statistical model using both simulated data and clinical evaluation studies in head and neck radiotherapy. Furthermore, the resulting SPP facilitates the measurement of the correlation between quantitative metrics and clinical experts' decision, and ultimately is able to guide the clinicians in selecting segmentation methods for radiotherapy.

Figure 1

Illustration of the volume overlap of auto-segmented ROI [D_i] and manual reference ROI (the ground truth [T_i]), and the definition of N_i and x_i.

Figure 2

Probability density function of Beta distribution with different parameters (α, β). The pdf plot shows the distribution of performance values in the interval [0, 1].

Figure 3

Some normal structures delineated for head and neck radiotherapy. The left and right parotid glands are in magenta color, the mandible is in green color, and the spinal cord is in read, surrounded by cervical vertebrae.

Figure 4

The MSEs of α and β estimations for different sizes of samples.

Figure 5

Ten examples of the simulated CT images with defined left and right parotid contours. This figure shows the axial view of these images in the same slice.

Figure 6

SPPs for direct mapping, affine registration, and deformable registration methods when they are applied to (a) left parotid contours and (b) right parotid contours. For better performance, the SPP curve should be narrow and sharp, and towards the high performance value. In this illustration, deformable registration shows a better performance than affine registration and direct mapping.

Figure 7

Auto-propagated contours for one patient with head-and-neck cancer. Original contours were drawn on the “Plan” CT and then were transformed automatically using deformable image registration to the daily CT images (“CT1”, “CT2”, …, “CT13”).

Figure 8

SPPs for contours from scratch and for modified contours. In this illustration, the “modified contours” shows a much better performance than the “contours from scratch”.

Figure 9

Comparison of the Beta distribution and Gaussian distribution for the parotid contours described in section 3.3. For the contouring from scratch, the performance distribution is similar for both Beta and Gaussian distribution; for modifying contours method, the Beta distribution is better than the Gaussian distribution since the effective range of Gaussian distribution is beyond the interval [0, 1].