A hierarchical fusion strategy of deep learning networks for detection and segmentation of hepatocellular carcinoma from computed tomography images

Abstract

Background: Automatic segmentation of hepatocellular carcinoma (HCC) on computed tomography (CT) scans is in urgent need to assist diagnosis and radiomics analysis. The aim of this study is to develop a deep learning based network to detect HCC from dynamic CT images.

Methods: Dynamic CT images of 595 patients with HCC were used. Tumors in dynamic CT images were labeled by radiologists. Patients were randomly divided into training, validation and test sets in a ratio of 5:2:3, respectively. We developed a hierarchical fusion strategy of deep learning networks (HFS-Net). Global dice, sensitivity, precision and F1-score were used to measure performance of the HFS-Net model.

Results: The 2D DenseU-Net using dynamic CT images was more effective for segmenting small tumors, whereas the 2D U-Net using portal venous phase images was more effective for segmenting large tumors. The HFS-Net model performed better, compared with the single-strategy deep learning models in segmenting small and large tumors. In the test set, the HFS-Net model achieved good performance in identifying HCC on dynamic CT images with global dice of 82.8%. The overall sensitivity, precision and F1-score were 84.3%, 75.5% and 79.6% per slice, respectively, and 92.2%, 93.2% and 92.7% per patient, respectively. The sensitivity in tumors < 2 cm, 2-3, 3-5 cm and > 5 cm were 72.7%, 92.9%, 94.2% and 100% per patient, respectively.

Conclusions: The HFS-Net model achieved good performance in the detection and segmentation of HCC from dynamic CT images, which may support radiologic diagnosis and facilitate automatic radiomics analysis.

Keywords: Computed tomography; Deep learning; Detection; Hepatocellular carcinoma; Segmentation.

Fig. 1

The HFS-Net data flow for liver and tumor segmentation. Stage I: Identify tumor’s longest axis in every slice of a case. Stage II: Accord tumor size by feeding the CT slice which the longest axis of tumors over 30 pixels in the slice for f_large computation and which the longest axis less than 30 pixels of tumors in the slice for f_small computation. Stage III: Combine venous phases of CT images and results of f_liver, f_size, f_large, and f_small as input of f_3D computation for getting final segmentation of tumors

Fig. 2

Performance of HFS-Net segmentation (left), detection (middle) and distribution of sizes of HCC (right)

Fig. 3

Examples of HFS-Net segmentation results from test dataset. A A 8.5 cm HCC representing large HCC (dice per case 88.1%). B A 4.6 cm HCC representing medium sized HCC (dice per case 80%). A A 8.5 cm HCC representing large HCC (dice per case 88.1%). C A 1.7 cm HCC representing BCLC stage 0 HCC (dice per case 83.6%). D A tumor was detected in the first three slices but missed in the marginal slice of the tumor. E A large tumor accurately segmented by HFS-Net, and a small tumor missed by radiologist labelling but detected by HFS-Net. The green line represents the radiologist’s label, the red line represents the output of HFS-Net for tumor margin, and the blue line represents the output of HFS-Net for liver margin