Mixed-Supervised Dual-Network for Medical Image Segmentation

Abstract

Deep learning based medical image segmentation models usually require large datasets with high-quality dense segmentations to train, which are very time-consuming and expensive to prepare. One way to tackle this difficulty is using the mixed-supervised learning framework, where only a part of data is densely annotated with segmentation label and the rest is weakly labeled with bounding boxes. The model is trained jointly in a multi-task learning setting. In this paper, we propose Mixed-Supervised Dual-Network (MSDN), a novel architecture which consists of two separate networks for the detection and segmentation tasks respectively, and a series of connection modules between the layers of the two networks. These connection modules are used to transfer useful information from the auxiliary detection task to help the segmentation task. We propose to use a recent technique called 'Squeeze and Excitation' in the connection module to boost the transfer. We conduct experiments on two medical image segmentation datasets. The proposed MSDN model outperforms multiple baselines.

Keywords: Dual-network; Medical image segmentation; Mixed-supervised learning; Multi-task learning; Squeeze-and-Excitation.

Fig.1.

Illustration of the Channel Squeeze and Spatial Excitation (sSE) architecture of Unary form (a) and Binary form (b).

Fig.2.

Structure of Mixed-Supervised Dual-Network (MSDN).

Fig.3.

(a) Original image. (b) Ground truth. (c) U-Net trained in full-supervised manner. (d) U-Net trained with only strongly-annotated data. (e) U-Net+Unary sSE. (f) MSDN-. (g),(h) Segmentation and detection results of Variant MS-Net. (i),(j) Segmentation and detection results of MSDN.