CNN-based automatic segmentations and radiomics feature reliability on contrast-enhanced ultrasound images for renal tumors

Abstract

Objective: To investigate the feasibility and efficiency of automatic segmentation of contrast-enhanced ultrasound (CEUS) images in renal tumors by convolutional neural network (CNN) based models and their further application in radiomic analysis.

Materials and methods: From 94 pathologically confirmed renal tumor cases, 3355 CEUS images were extracted and randomly divided into training set (3020 images) and test set (335 images). According to the histological subtypes of renal cell carcinoma, the test set was further split into clear cell renal cell carcinoma (ccRCC) set (225 images), renal angiomyolipoma (AML) set (77 images) and set of other subtypes (33 images). Manual segmentation was the gold standard and serves as ground truth. Seven CNN-based models including DeepLabV3+, UNet, UNet++, UNet3+, SegNet, MultilResUNet and Attention UNet were used for automatic segmentation. Python 3.7.0 and Pyradiomics package 3.0.1 were used for radiomic feature extraction. Performance of all approaches was evaluated by the metrics of mean intersection over union (mIOU), dice similarity coefficient (DSC), precision, and recall. Reliability and reproducibility of radiomics features were evaluated by the Pearson coefficient and the intraclass correlation coefficient (ICC).

Results: All seven CNN-based models achieved good performance with the mIOU, DSC, precision and recall ranging between 81.97%-93.04%, 78.67%-92.70%, 93.92%-97.56%, and 85.29%-95.17%, respectively. The average Pearson coefficients ranged from 0.81 to 0.95, and the average ICCs ranged from 0.77 to 0.92. The UNet++ model showed the best performance with the mIOU, DSC, precision and recall of 93.04%, 92.70%, 97.43% and 95.17%, respectively. For ccRCC, AML and other subtypes, the reliability and reproducibility of radiomic analysis derived from automatically segmented CEUS images were excellent, with the average Pearson coefficients of 0.95, 0.96 and 0.96, and the average ICCs for different subtypes were 0.91, 0.93 and 0.94, respectively.

Conclusion: This retrospective single-center study showed that the CNN-based models had good performance on automatic segmentation of CEUS images for renal tumors, especially the UNet++ model. The radiomics features extracted from automatically segmented CEUS images were feasible and reliable, and further validation by multi-center research is necessary.

Keywords: UNet; automatic segmentation; contrast-enhanced ultrasound; deep learning; renal tumors.

Figure 1

Flowchart of the study with image preprocessing, image segmentation, and evaluation methods.

Figure 2

(A) The network of a classic U-Net model, where X^i,j is the convolution block. The input of each X^i,j is concatenated from the up-sampling of X^i+1,j-1 from the earlier convolution layer of the same dense block and all of X^i,k (k< j) from the same pyramid level. (B) The process of automatic segmentation by the algorithm.

Figure 3

Typical target area segmented by manual delineation, DeepLabV3+, UNet, SegNet, MiltiResUNet, Att_UNet, UNet3+ and UNet++ segmentation models on CEUS images for renal tumors, respectively.

Figure 4

Heat maps of Pearson correlation and intraclass correlation coefficients for radiomics features extracted from CNN-based automatic segmentations.

Figure 5

Heat maps of Pearson correlation and intraclass correlation coefficients for radiomics features extracted from manual segmentation and U-Net++ models based on automatic segmentation in different subtypes of renal tumor.