Multi-needle Localization with Attention U-Net in US-guided HDR Prostate Brachytherapy

Abstract

Purpose: Ultrasound (US)-guided high dose rate (HDR) prostate brachytherapy requests the clinicians to place HDR needles (catheters) into the prostate gland under transrectal US (TRUS) guidance in the operating room. The quality of the subsequent radiation treatment plan is largely dictated by the needle placements, which varies upon the experience level of the clinicians and the procedure protocols. Real-time plan dose distribution, if available, could be a vital tool to provide more subjective assessment of the needle placements, hence potentially improving the radiation plan quality and the treatment outcome. However, due to low signal-to-noise ratio (SNR) in US imaging, real-time multi-needle segmentation in 3D TRUS, which is the major obstacle for real-time dose mapping, has not been realized to date. In this study, we propose a deep learning-based method that enables accurate and real-time digitization of the multiple needles in the 3D TRUS images of HDR prostate brachytherapy.

Methods: A deep learning model based on the U-Net architecture was developed to segment multiple needles in the 3D TRUS images. Attention gates were considered in our model to improve the prediction on the small needle points. Furthermore, the spatial continuity of needles was encoded into our model with total variation (TV) regularization. The combined network was trained on 3D TRUS patches with the deep supervision strategy, where the binary needle annotation images were provided as ground truth. The trained network was then used to localize and segment the HDR needles for a new patient's TRUS images. We evaluated our proposed method based on the needle shaft and tip errors against manually defined ground truth and compared our method with other state-of-art methods (U-Net and deeply supervised attention U-Net).

Results: Our method detected 96% needles of 339 needles from 23 HDR prostate brachytherapy patients with 0.290 ± 0.236 mm at shaft error and 0.442 ± 0.831 mm at tip error. For shaft localization, our method resulted in 96% localizations with less than 0.8 mm error (needle diameter is 1.67 mm), while for tip localization, our method resulted in 75% needles with 0 mm error and 21% needles with 2 mm error (TRUS image slice thickness is 2 mm). No significant difference is observed (P = 0.83) on tip localization between our results with the ground truth. Compared with U-Net and deeply supervised attention U-Net, the proposed method delivers a significant improvement on both shaft error and tip error (P < 0.05).

Conclusions: We proposed a new segmentation method to precisely localize the tips and shafts of multiple needles in 3D TRUS images of HDR prostate brachytherapy. The 3D rendering of the needles could help clinicians to evaluate the needle placements. It paves the way for the development of real-time plan dose assessment tools that can further elevate the quality and outcome of HDR prostate brachytherapy.

Keywords: deep learning; multi-needle localization; prostate brachytherapy; total variation regularization; ultrasound images.

Fig. 1.

An example of US slice with needles (a), its annotations (b), and its mask image (c).

Fig. 2.

The schematic workflow of the proposed method.

Fig. 3.

A needle-directional profile of 3D needle US image that has in total 29 slices. The yellow lines are needles and the red arrows indicate noise.

Fig. 4.

The 2D visualization of needle detections in US slices. The rows from top to down corresponds to the US images, ground truth, U-Net, DSA U-Net, and the proposed method, respectively. The columns from left to right, respectively, corresponds to the 1th, 7th, 14th, 21th, and 28th slice of the US images. The yellow circles show the incorrect localizations, and the blue triangles show the missed localizations.

Fig. 5.

The 3D visualization of the detected needles. Different colors indicate different needles, where the black arrows indicate the incorrect detections.

Fig. 6.

Needle shaft localization error on all patients with the three methods.

Fig. 7.

The error distributions of the shaft localization center with the three methods.

Fig. 8.

The tip localization error distributions in terms of standard difference (a) and absolute difference (b).

Fig. 9.

The distribution of the tip localization error with the three methods.