A study of generalization and compatibility performance of 3D U-Net segmentation on multiple heterogeneous liver CT datasets

Abstract

Background: Most existing algorithms have been focused on the segmentation from several public Liver CT datasets scanned regularly (no pneumoperitoneum and horizontal supine position). This study primarily segmented datasets with unconventional liver shapes and intensities deduced by contrast phases, irregular scanning conditions, different scanning objects of pigs and patients with large pathological tumors, which formed the multiple heterogeneity of datasets used in this study.

Methods: The multiple heterogeneous datasets used in this paper includes: (1) One public contrast-enhanced CT dataset and one public non-contrast CT dataset; (2) A contrast-enhanced dataset that has abnormal liver shape with very long left liver lobes and large-sized liver tumors with abnormal presets deduced by microvascular invasion; (3) One artificial pneumoperitoneum dataset under the pneumoperitoneum and three scanning profiles (horizontal/left/right recumbent position); (4) Two porcine datasets of Bama type and domestic type that contains pneumoperitoneum cases but with large anatomy discrepancy with humans. The study aimed to investigate the segmentation performances of 3D U-Net in: (1) generalization ability between multiple heterogeneous datasets by cross-testing experiments; (2) the compatibility when hybrid training all datasets in different sampling and encoder layer sharing schema. We further investigated the compatibility of encoder level by setting separate level for each dataset (i.e., dataset-wise convolutions) while sharing the decoder.

Results: Model trained on different datasets has different segmentation performance. The prediction accuracy between LiTS dataset and Zhujiang dataset was about 0.955 and 0.958 which shows their good generalization ability due to that they were all contrast-enhanced clinical patient datasets scanned regularly. For the datasets scanned under pneumoperitoneum, their corresponding datasets scanned without pneumoperitoneum showed good generalization ability. Dataset-wise convolution module in high-level can improve the dataset unbalance problem. The experimental results will facilitate researchers making solutions when segmenting those special datasets.

Conclusions: (1) Regularly scanned datasets is well generalized to irregularly ones. (2) The hybrid training is beneficial but the dataset imbalance problem always exits due to the multi-domain homogeneity. The higher levels encoded more domain specific information than lower levels and thus were less compatible in terms of our datasets.

Keywords: Dataset-wise convolution; Generalization; Liver segmentation; U-Net.

Fig. 1

The pipeline of the method and the segmentation networks used in this study. The proposed dataset-wise convolution module (DCM) sets separate level convolutions for the eleven datasets. Convolution layers in the encoder was leveled by the size of their output features maps and numbered by down-sampling time. GELN_DCM 3D U-Net set DCM at those levels greater or equal to Level (GEL) N (N ranges from zero to five) in the encoder of 3D U-Net

Fig. 2

Multiple liver CT datasets in different scanning conditions—A public contrast-enhanced CT liver tumor dataset from five developed countries in LiTS Challenge, B public non-contrast CT normal liver segmentation dataset in Anatomy3 Challenge, C clinical patients with long left liver lobes (case #1 and #3) and large and intensity-varied (low or high) liver tumor changes from Zhujiang Hospital in China, D non-contrast CT dataset from real patients scanned regularly (APP_0) and irregularly (APP_1-3) with different scanning profiles under artificial pneumoperitoneum (APP) pressure, E non-contrast Bama Pig (PB) CT dataset and F contrast-enhanced domestic pig (PD) CT with (PB1 & PD1) or without (PB0 & PD0) pneumoperitoneum pressure

Fig. 3

Bar chart of comparison results measured with DSC for datasets scanned under pneumoperitoneum which are predicted by their corresponding dataset scanned regularly (without pneumoperitoneum) and two-fold model respectively. Dataset scanned without pneumoperitoneum showed good generalization ability

Fig. 4

Bar charts of comparison results measured with DSC for eleven datasets grouped by different sampling strategies when training all datasets together by two-fold (the non-hybrid training schema fold 2 and fold 5 were used as baseline). The dataset-balance extent of sample strategy decreased from DOS > RSD > RS. Most datasets benefit from the hybrid training except the unbalanced dataset. LiTS and Porcine dataset was unbalanced dataset in DOS and RS strategy respectively. Zhujiang dataset cannot benefit from hybrid training in any sample strategy

Fig. 5

Bar charts of comparison results measured with DSC for eleven datasets tested by hybrid-training models with different encoder layer sharing schema. FullyShare was another name of the DOS result in Fig. 4. GELN_DCM denotes segmentation from GELN_DCM 3D U-Net in Fig. 2. The blue triangle denotes an obvious accuracy-decreased stagnation level, which suggested that the stagnation level and the lower levels should be shared and thus were more compatible. The GEL5-DCM can improve the unbalanced datasets’ accuracy while not reduce others’, which suggested that the final level of the encoder was the least compatible

Fig. 6

Visualization segmentation results of three comparison methods for hard examples by task. The blue, red and green line respectively show the segmentation results by reference segmentation, the simple 3D U-Net in two-fold non-hybrid training schema and the 3D U-Net in hybrid training with DOS sampling strategy and GEL_DCM layer sharing schema

Fig. 7

Box charts of comparison results measured with DSC for eleven datasets segmented by five-folds (white), two-fold (grey) and two-fold using LiTS Pre-trained model (red) respectively. For most datasets, there shows no great significance between the five-fold and two-fold results