Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Nov 17:28:100515.
doi: 10.1016/j.phro.2023.100515. eCollection 2023 Oct.

Clinical Implementation and Evaluation of Auto-Segmentation Tools for Multi-Site Contouring in Radiotherapy

Gerd Heilemann  1 , Martin Buschmann  1 , Wolfgang Lechner  1 , Vincent Dick  1 , Franziska Eckert  1 , Martin Heilmann  1 , Harald Herrmann  1 , Matthias Moll  1 , Johannes Knoth  1 , Stefan Konrad  1 , Inga-Malin Simek  1 , Christopher Thiele  1 , Alexandru Zaharie  1 , Dietmar Georg  1 , Joachim Widder  1 , Petra Trnkova  1
Affiliations

Clinical Implementation and Evaluation of Auto-Segmentation Tools for Multi-Site Contouring in Radiotherapy

Gerd Heilemann et al. Phys Imaging Radiat Oncol. .

Abstract

Background and purpose: Tools for auto-segmentation in radiotherapy are widely available, but guidelines for clinical implementation are missing. The goal was to develop a workflow for performance evaluation of three commercial auto-segmentation tools to select one candidate for clinical implementation.

Materials and methods: One hundred patients with six treatment sites (brain, head-and-neck, thorax, abdomen, and pelvis) were included. Three sets of AI-based contours for organs-at-risk (OAR) generated by three software tools and manually drawn expert contours were blindly rated for contouring accuracy. The dice similarity coefficient (DSC), the Hausdorff distance, and a dose/volume evaluation based on the recalculation of the original treatment plan were assessed. Statistically significant differences were tested using the Kruskal-Wallis test and the post-hoc Dunn Test with Bonferroni correction.

Results: The mean DSC scores compared to expert contours for all OARs combined were 0.80 ± 0.10, 0.75 ± 0.10, and 0.74 ± 0.11 for the three software tools. Physicians' rating identified equivalent or superior performance of some AI-based contours in head (eye, lens, optic nerve, brain, chiasm), thorax (e.g., heart and lungs), and pelvis and abdomen (e.g., kidney, femoral head) compared to manual contours. For some OARs, the AI models provided results requiring only minor corrections. Bowel-bag and stomach were not fit for direct use. During the interdisciplinary discussion, the physicians' rating was considered the most relevant.

Conclusion: A comprehensive method for evaluation and clinical implementation of commercially available auto-segmentation software was developed. The in-depth analysis yielded clear instructions for clinical use within the radiotherapy department.

Keywords: Auto-segmentation; Deep Learning; Radiotherapy; Segmentation.

PubMed Disclaimer

Conflict of interest statement

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Figures

Fig. 1
Fig. 1
Flowchart of the implementation and selection of auto-segmentation software in clinical practice.
Fig. 2
Fig. 2
Results of the DSC, HD (in mm), Dmean and D1% comparison between software tools A, B and C for the thorax, abdomen, and pelvis region. Dmean and D1% are given as relative differences (%) compared to the original plan. The roman numerals indicate the classification according to chapter 2.5 in categories of class I, class II and class III.
Fig. 3
Fig. 3
Results of the the DSC, HD (in mm), Dmean and D1% comparison between software tools A, B and C for the central nervous system, head and neck, and upper digestive tract. Dmean and D1% are given as relative differences (%) compared to the original plan. The roman numerals indicate the classification according to chapter 2.5 in categories of class I, class II and class III.
See this image and copyright information in PMC

References

    1. Nelms B.E., Tomé W.A., Robinson G., Wheeler J. Variations in the contouring of organs at risk: Test case from a patient with oropharyngeal cancer. Int J Radiat Oncol Biol Phys. 2012;82:368–378. doi: 10.1016/j.ijrobp.2010.10.019. - DOI - PubMed
    1. Boero I.J., Paravati A.J., Xu B., Cohen E.E.W., Mell L.K., Le Q.-T., et al. Importance of Radiation Oncologist Experience Among Patients With Head-and-Neck Cancer Treated With Intensity-Modulated Radiation Therapy. J Clin Oncol. 2016;34:684–690. doi: 10.1200/JCO.2015.63.9898. - DOI - PMC - PubMed
    1. Sherer M.V., Lin D., Elguindi S., Duke S., Tan L.T., Cacicedo J., et al. Metrics to evaluate the performance of auto-segmentation for radiation treatment planning: A critical review. Radiother Oncol. 2021;160:185–191. doi: 10.1016/j.radonc.2021.05.003. - DOI - PMC - PubMed
    1. Thor M., Apte A., Haq R., Iyer A., LoCastro E., Deasy J.O. Using Auto-Segmentation to Reduce Contouring and Dose Inconsistency in Clinical Trials: The Simulated Impact on RTOG 0617. Int J Radiat Oncol Biol Phys. 2021;109:1619–1626. doi: 10.1016/j.ijrobp.2020.11.011. - DOI - PMC - PubMed
    1. Han X., Hoogeman M.S., Levendag P.C., Hibbard L.S., Teguh D.N., Voet P., et al. Atlas-Based Auto-segmentation of Head and Neck CT Images. Med Image Comput Comput Assist Interv. 2008;11:434–441. doi: 10.1007/978-3-540-85990-1_52. - DOI - PubMed

LinkOut - more resources