The suitability of common metrics for assessing parotid and larynx autosegmentation accuracy

Abstract

Contouring structures in the head and neck is time-consuming, and automatic seg-mentation is an important part of an adaptive radiotherapy workflow. Geometric accuracy of automatic segmentation algorithms has been widely reported, but there is no consensus as to which metrics provide clinically meaningful results. This study investigated whether geometric accuracy (as quantified by several commonly used metrics) was associated with dosimetric differences for the parotid and larynx, comparing automatically generated contours against manually drawn ground truth contours. This enabled the suitability of different commonly used metrics to be assessed for measuring automatic segmentation accuracy of the parotid and larynx. Parotid and larynx structures for 10 head and neck patients were outlined by five clinicians to create ground truth structures. An automatic segmentation algorithm was used to create automatically generated normal structures, which were then used to create volumetric-modulated arc therapy plans. The mean doses to the automatically generated structures were compared with those of the corresponding ground truth structures, and the relative difference in mean dose was calculated for each structure. It was found that this difference did not correlate with the geometric accuracy provided by several metrics, notably the Dice similarity coefficient, which is a commonly used measure of spatial overlap. Surface-based metrics provided stronger correlation and are, therefore, more suitable for assessing automatic seg-mentation of the parotid and larynx.

Figure 1

DSC and DTA. DSC measures the spatial overlap between two volumes, and DTA describes the shortest distance between two surfaces for a specific point.

Figure 2

Dosimetric interobserver variation for the parotids. Box plot showing the interobserver variation in dosimetric accuracy relative to the STAPLE contours for the parotid glands. Red boxes indicate right hand parotid glands and blue boxes indicate left hand glands. The boxes indicate the interquartile range, the whiskers indicate the minimum and maximum variation, and the horizontal lines indicate the median accuracy of the five clinician contours. The mean dosimetric accuracy of the automatically generated contours is indicated by the circles.

Figure 3

Dosimetric interobserver variation for the larynx. Box plot showing the interobserver variation in dosimetric accuracy relative to the STAPLE contours for the larynx. The boxes indicate the interquartile range, the whiskers indicate the minimum and maximum variation, and the horizontal lines indicate the median accuracy of the five clinician contours. The mean dosimetric accuracy of the automatically generated contours is indicated by the circles.

Figure 4

Scatter plots showing the relationship between dosimetric and geometric accuracy for the parotid. The left hand plot shows the relationship for the centroid separation $(R=0.82)$, and the right hand plot shows the relationship for DSC $(R=-0.35)$. Lines of best fit are also shown.