State-of-the-Art Deep Learning Methods for Objects Detection in Remote Sensing Satellite Images

Abstract

Introduction: Object detection in remotely sensed satellite images is critical to socio-economic, bio-physical, and environmental monitoring, necessary for the prevention of natural disasters such as flooding and fires, socio-economic service delivery, and general urban and rural planning and management. Whereas deep learning approaches have recently gained popularity in remotely sensed image analysis, they have been unable to efficiently detect image objects due to complex landscape heterogeneity, high inter-class similarity and intra-class diversity, and difficulty in acquiring suitable training data that represents the complexities, among others. Methods: To address these challenges, this study employed multi-object detection deep learning algorithms with a transfer learning approach on remotely sensed satellite imagery captured on a heterogeneous landscape. In the study, a new dataset of diverse features with five object classes collected from Google Earth Engine in various locations in southern KwaZulu-Natal province in South Africa was used to evaluate the models. The dataset images were characterized with objects that have varying sizes and resolutions. Five (5) object detection methods based on R-CNN and YOLO architectures were investigated via experiments on our newly created dataset. Conclusions: This paper provides a comprehensive performance evaluation and analysis of the recent deep learning-based object detection methods for detecting objects in high-resolution remote sensing satellite images. The models were also evaluated on two publicly available datasets: Visdron and PASCAL VOC2007. Results showed that the highest detection accuracy of the vegetation and swimming pool instances was more than 90%, and the fastest detection speed 0.2 ms was observed in YOLOv8.

Keywords: R-CNN; YOLO; object detection; remote sensing; satellite images.

Figure 1

Architectural diagram for the proposed methodology.

Figure 2

A diagram showing extracted region for dataset acquisition from larger map view [40].

Figure 3

Region of Interest (ROI) identification for dataset collection [40].

Figure 4

Example of images from Training dataset.

Figure 5

Labeling images for objects detection.

Figure 6

An illustration of backbone, neck and head composition in Generalized Object Detection Systems Architecture.

Figure 7

Detecting dense objects from sample satellite image testing datasets; (i) Detection of Swimming Pools, Vegetation, and Residences; (ii) Detection of dense objects such as Vegetation; (iii) Detection of Vegetation and Swimming Pools.

Figure 8

Densely Distributed objects detection from sample satellite image testing datasets; (i) Detecting clustered vegetation objects; (ii) Detecting clustered vegetation and roads objects; (iii) Detecting clustered residence, shorelines and roads objects.

Figure 9

Evenly distributed objects detection from sample satellite testing datasets; (i) Detecting swimming pools, residence and vegetation objects; (ii) Detecting vegetation, swimming pools and residence objects; (iii) Detecting residence, swimming pools and roads objects.

Figure 10

Detecting evenly and densely distributed objects from sample satellite image testing datasets; (i) Detecting densely distributed vegetation and evenly distributed residence objects; (ii) Detecting evenly distributed residence and vegetation objects; (iii) Detecting evenly distributed residence, vegetation and roads objects.

Figure 11

More Evenly Distributed objects detection from sample satellite images (i) Detecting evenly distributed residence, roads, vegetation and swimming pools objects; (ii) Detecting evenly distributed roads, residence and vegetation objects; (iii) Detecting slightly and densely distributed shorelines, vegetation and roads objects.

Figure 12

Performance evaluation curves of YOLOv8 on the proposed dataset; First row presents training curves for box regression loss, class loss, precision, and recall; Second row presents validation curves for box regression loss, class loss, mean average precision:50, and mean average precision: 50:95 dataset.