GLE-net: global-local information enhancement for semantic segmentation of remote sensing images

Abstract

Remote sensing (RS) images contain a wealth of information with expansive potential for applications in image segmentation. However, Convolutional Neural Networks (CNN) face challenges in fully harnessing the global contextual information. Leveraging the formidable capabilities of global information modeling with Swin-Transformer, a novel RS images segmentation model with CNN (GLE-Net) was introduced. This integration gives rise to a revamped encoder structure. The subbranch initiates the process by extracting features at varying scales within the RS images using the Multiscale Feature Fusion Module (MFM), acquiring rich semantic information, discerning localized finer features, and adeptly handling occlusions. Subsequently, Feature Compression Module (FCM) is introduced in main branch to downsize the feature map, effectively reducing information loss while preserving finer details, enhancing segmentation accuracy for smaller targets. Finally, we integrate local features and global features through Spatial Information Enhancement Module (SIEM) for comprehensive feature modeling, augmenting the segmentation capabilities of model. We performed experiments on public datasets provided by ISPRS, yielding notably remarkable experimental outcomes. This underscores the substantial potential of our model in the realm of RS image segmentation within the context of scientific research.

Keywords: Multiscale feature; Remote sening; Swin-transformer convolutional neural networks.

Fig. 1

Figure of RS imagery, shadows will further reduce the segmentation accuracy of objects that are originally very similar to the ground objects.

Fig. 2

Overall network structure of GLE-Net.

Fig. 3

Specific structure of the MFM.

Fig. 4

Structure of our proposed SIEM.

Fig. 5

Structure of the FCM.

Fig. 6

Comparison of ablation experiments. (a) Image. (b) (GT). (c) (ST). (d) (ST+MFM). (e) (ST+SIEM). (f) (ST+ FCM).

Fig. 7

Comparison of segmentation with other methods on the Vaihingen dataset. (a) Image. (b) (GT). (c) (FCN). (d) (U-Net). (e) (Deeplab V3+). (f) (PSPNet). (g) (Trans-UNet). (h) (Swin-UNet). (i) (GLE-Net).

Fig. 8

Comparison of segmentation with other methods on the Potsdam dataset. (a) Image. (b) (GT). (c) (FCN). (d) (U-Net). (e) (Deeplab V3+). (f) (PSPNet). (g) (Trans-UNet). (h) (Swin-UNet). (i) (GLE-Net).