Deep learning-based digitization of prostate brachytherapy needles in ultrasound images

Abstract

Purpose: To develop, and evaluate the performance of, a deep learning-based three-dimensional (3D) convolutional neural network (CNN) artificial intelligence (AI) algorithm aimed at finding needles in ultrasound images used in prostate brachytherapy.

Methods: Transrectal ultrasound (TRUS) image volumes from 1102 treatments were used to create a clinical ground truth (CGT) including 24422 individual needles that had been manually digitized by medical physicists during brachytherapy procedures. A 3D CNN U-net with 128 × 128 × 128 TRUS image volumes as input was trained using 17215 needle examples. Predictions of voxels constituting a needle were combined to yield a 3D linear function describing the localization of each needle in a TRUS volume. Manual and AI digitizations were compared in terms of the root-mean-square distance (RMSD) along each needle, expressed as median and interquartile range (IQR). The method was evaluated on a data set including 7207 needle examples. A subgroup of the evaluation data set (n = 188) was created, where the needles were digitized once more by a medical physicist (G1) trained in brachytherapy. The digitization procedure was timed.

Results: The RMSD between the AI and CGT was 0.55 (IQR: 0.35-0.86) mm. In the smaller subset, the RMSD between AI and CGT was similar (0.52 [IQR: 0.33-0.79] mm) but significantly smaller (P < 0.001) than the difference of 0.75 (IQR: 0.49-1.20) mm between AI and G1. The difference between CGT and G1 was 0.80 (IQR: 0.48-1.18) mm, implying that the AI performed as well as the CGT in relation to G1. The mean time needed for human digitization was 10 min 11 sec, while the time needed for the AI was negligible.

Conclusions: A 3D CNN can be trained to identify needles in TRUS images. The performance of the network was similar to that of a medical physicist trained in brachytherapy. Incorporating a CNN for needle identification can shorten brachytherapy treatment procedures substantially.

Keywords: brachytherapy; deep learning; high-dose-rate; image segmentation; needle digitization.

Fig 1

Illustration of the prostate in the sagittal plane with needles in a brachytherapy treatment setting for prostate cancer. The transrectal ultrasound probe is used to visualize the prostate gland and the needles in place. [Color figure can be viewed at wileyonlinelibrary.com]

Fig 2

Ultrasound images encompassing the complete prostate gland with margins, acquired after a sweep of the transrectal ultrasound, yielding a three‐dimensional volume including 20 needles (highlighted with red lines). [Color figure can be viewed at wileyonlinelibrary.com]

Fig 3

Graphical illustration of the applied three‐dimensional U‐net with a 128 × 128 × 128 TRUS image volume as input and the needle localizations in the volume as output. Convolutions (blue arrows), maxpooling (red arrows), and transposed convolutions (green arrows) were performed using 3 × 3 × 3 kernels. The gray horizontal arrows illustrate the concatenation of data of feature maps (feat) from the contraction path to the expansion path. [Color figure can be viewed at wileyonlinelibrary.com]

Fig 4

Logarithm histograms for (a) ${\text{RMSD}}_{\mathrm{AI},\mathrm{CGT},\mathrm{n}=7207}$, (b) ${\text{RMSD}}_{\mathrm{CGT},\mathrm{G}1,\mathrm{n}=188}$, (c) ${\text{RMSD}}_{\mathrm{AI},\mathrm{CGT},\mathrm{n}=188}$, and (d) ${\text{RMSD}}_{\mathrm{AI},\mathrm{G}1,\mathrm{n}=188}$. A normal distribution adapted to the data is shown as a red curve for each histogram. [Color figure can be viewed at wileyonlinelibrary.com]

Fig 5

Visual comparison between artificial intelligence (AI) digitization (left panel) and the clinical ground truth (right panel). The AI digitization is represented by expectation values with an isosurface of 0.1 [Color figure can be viewed at wileyonlinelibrary.com]