Automatic segmentation of liver structures in multi-phase MRI using variants of nnU-Net and Swin UNETR

Abstract

Accurate segmentation of the liver parenchyma, portal veins, hepatic veins, and lesions from MRI is important for hepatic disease monitoring and treatment. Multi-phase contrast enhanced imaging is superior in distinguishing hepatic structures compared to single-phase approaches, but automated approaches for detailed segmentation of hepatic structures are lacking. This study evaluates deep learning architectures for segmenting liver structures from multi-phase Gd-EOB-DTPA-enhanced T1-weighted VIBE MRI scans. We utilized 458 T1-weighted VIBE scans of pathological livers, with 78 manually labeled for liver parenchyma, hepatic and portal veins, aorta, lesions, and ascites. An additional dataset of 47 labeled subjects was used for cross-scanner evaluation. Three models were evaluated using nested cross-validation: the conventional nnU-Net, the ResEnc nnU-Net, and the Swin UNETR. The late arterial phase was identified as the optimal fixed phase for co-registration. Both nnU-Net variants outperformed Swin UNETR across most tasks. The conventional nnU-Net achieved the highest segmentation performance for liver parenchyma (DSC: 0.97; 95% CI 0.97, 0.98), portal vein (DSC: 0.83; 95% CI 0.80, 0.87), and hepatic vein (DSC: 0.78; 95% CI 0.77, 0.80). Lesion and ascites segmentation proved challenging for all models, with the conventional nnU-Net performing best. This study demonstrates the effectiveness of deep learning, particularly nnU-Net variants, for detailed liver structure segmentation from multi-phase MRI. The developed models and preprocessing pipeline offer potential for improved liver disease assessment and surgical planning in clinical practice.

Fig. 1

Arterial and portal venous phases from the T1-VIBE sequence for one subject before co-registration. The arterial phase is shown in the (left) images, and the portal venous phase in the (right). The (top row) displays the sagittal view. The (bottom row) shows the axial orientation of the same MRI. Images in the same row depict the exact same section. Due to variations in the breathing cycle, the abdominal organs do not remain in a fixed position. Brown reference lines aid visualization. Vertical lines in the top images indicate the moving abdominal border: the right line marks the border in the arterial phase, while the left line marks the border in the portal venous phase, where less air was exhaled. Horizontal lines in these images illustrate the movement of the heart (top line) and vessels (bottom lines) between phases due to breathing. Yellow arrows in the axial images point to identical spatial positions where the hepatic veins are visible in the portal venous phase. The horizontal lines indicate the borders of the liver and spleen during the arterial phase and demonstrate how they shift due to breathing.

Fig. 2

All images show the same axial section of a patient with liver cirrhosis and ascites for the late arterial phase. First row is the given image with no annotations. Labels in the other sections are blue (liver), orange (portal vein), red (hepatic vein), mauve (ascites), green (lesion), and yellow (abdominal aorta). The bottom row from left to right depicts the ground-truth annotations, standard nnU-Net’s, ResEnc nnU-Net’s and SWIN UNETR’s segmentations. Arrows, ellipses and rectangular boxes highlight major differences between the segmentations from the models and the ground-truth annotations.

Fig. 3

All images show the same axial section of a patient with liver disease for the portalvenous phase. First row is the given image with no annotations. Labels in the other images are blue (liver), orange (portal vein), red (hepatic vein) and yellow (abdominal aorta). The bottom row from left to right depicts the ground-truth annotations, standard nnU-Net’s, ResEnc nnU-Net’s and SWIN UNETR’s segmentations. Arrows and ellipses highlight major differences between the segmentations from the models and the ground-truth annotations. The pink rectangular boxes show hepatic and portal veins in the ground-truth annotations that all of the architectures missed.