Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2004 Feb;11(2):178-89.
doi: 10.1016/s1076-6332(03)00671-8.

Statistical validation of image segmentation quality based on a spatial overlap index

Kelly H Zou  1 , Simon K Warfield, Aditya Bharatha, Clare M C Tempany, Michael R Kaus, Steven J Haker, William M Wells 3rd, Ferenc A Jolesz, Ron Kikinis
Affiliations

Statistical validation of image segmentation quality based on a spatial overlap index

Kelly H Zou et al. Acad Radiol. 2004 Feb.

Abstract

Rationale and objectives: To examine a statistical validation method based on the spatial overlap between two sets of segmentations of the same anatomy.

Materials and methods: The Dice similarity coefficient (DSC) was used as a statistical validation metric to evaluate the performance of both the reproducibility of manual segmentations and the spatial overlap accuracy of automated probabilistic fractional segmentation of MR images, illustrated on two clinical examples. Example 1: 10 consecutive cases of prostate brachytherapy patients underwent both preoperative 1.5T and intraoperative 0.5T MR imaging. For each case, 5 repeated manual segmentations of the prostate peripheral zone were performed separately on preoperative and on intraoperative images. Example 2: A semi-automated probabilistic fractional segmentation algorithm was applied to MR imaging of 9 cases with 3 types of brain tumors. DSC values were computed and logit-transformed values were compared in the mean with the analysis of variance (ANOVA).

Results: Example 1: The mean DSCs of 0.883 (range, 0.876-0.893) with 1.5T preoperative MRI and 0.838 (range, 0.819-0.852) with 0.5T intraoperative MRI (P < .001) were within and at the margin of the range of good reproducibility, respectively. Example 2: Wide ranges of DSC were observed in brain tumor segmentations: Meningiomas (0.519-0.893), astrocytomas (0.487-0.972), and other mixed gliomas (0.490-0.899).

Conclusion: The DSC value is a simple and useful summary measure of spatial overlap, which can be applied to studies of reproducibility and accuracy in image segmentation. We observed generally satisfactory but variable validation results in two clinical applications. This metric may be adapted for similar validation tasks.

PubMed Disclaimer

Figures

Figure 1
Figure 1
The Dice similarity coefficient representing spatial overlap and reproducibility, where DSC = 2 (intersected region)/(sum of region A and region B).
Figure 2
Figure 2
Prostate T2-weighted preoperative 1.5T MRI (a) and intraoperative 0.5T MRI (b) from the same brachytherapy patient, along with manual segmentations of the PZ on these 1.5T (c) and 0.5T (d) images, in detail. The DSC and reproducibility of PZ segmentations were significantly higher on the preoperative 1.5T image.
Figure 3
Figure 3
Boxplots of all DSCs based on all pair-wise repeated segmentations of the prostate PZ using preoperative 1.5T images.
Figure 4
Figure 4
Boxplots of all DSCs based on all pair-wise repeated segmentations of the prostate PZ using intraoperative 0.5T images.
See this image and copyright information in PMC

References

    1. Bonar DC, Schaper KA, Anderson JR, Rottenberg DA, Strother SC. Graphical analysis of MR feature space for measurement of CSF, gray matter, and white-matter volumes. J Comput Assist Tomogr. 1993;17:461–470. - PubMed
    1. Warfield SK, Westin CF, Guttmann CRG, Albert M, Jolesz FA, Kikinis R. Fractional segmentation of white matter. In: Proceedings of Second International Conference on Medical Imaging Computing and Computer Assisted Interventions, Sept 19–22, 1999, Cambridge, UK. New York: Springer, 62–71.
    1. Zou KH, Wells M III, Kaus MR, Kikinis R, Jolesz FA, Warfield SK. Statistical validation of automated probabilistic fractional segmentation against composite latent expert gold standard in MR imaging of brain tumors. In: Proceedings of 5th International Conference on Medical Imaging Computing and Computer Assisted Interventions, Sept 25–28, 2002, Tokyo, Japan. Berlin: Springer-Verlag, 315–322.
    1. Grabowski TJ, Frank RJ, Szumski NR, Brown CK, Damasio H. Validation of partial tissue segmentation of single-channel magnetic resonance images of the brain. Neuroimage. 2000;12:640 –656. - PubMed
    1. Choi HS, Haynor DR, Kim Y. Partial volume tissue classification of multi-channel magnetic resonance images – a mixed model. IEEE Trans Med Imag. 1991;10:295–407. - PubMed

Publication types

LinkOut - more resources