Metrics to evaluate the performance of auto-segmentation for radiation treatment planning: A critical review

Abstract

Advances in artificial intelligence-based methods have led to the development and publication of numerous systems for auto-segmentation in radiotherapy. These systems have the potential to decrease contour variability, which has been associated with poor clinical outcomes and increased efficiency in the treatment planning workflow. However, there are no uniform standards for evaluating auto-segmentation platforms to assess their efficacy at meeting these goals. Here, we review the most frequently used evaluation techniques which include geometric overlap, dosimetric parameters, time spent contouring, and clinical rating scales. These data suggest that many of the most commonly used geometric indices, such as the Dice Similarity Coefficient, are not well correlated with clinically meaningful endpoints. As such, a multi-domain evaluation, including composite geometric and/or dosimetric metrics with physician-reported assessment, is necessary to gauge the clinical readiness of auto-segmentation for radiation treatment planning.

Keywords: Auto-segmentation; Contouring; Quality assurance; Treatment planning.

Fig. 1.

Overview of metrics used for contour evaluation.

Fig. 2.

Examples of Geometric-Dosimetric discordance. On the left, two contours of the left anterior descending artery have almost no overlap but both structures receive a nearly identical dose (Figure reprinted with permission from reference [52]). On the right, two small bowel contours have excellent geometric agreement but disagreement within a high dose gradient region would result in a higher Dmax for the red contour.