OS-PSO: A Modified Ratio of Exponentially Weighted Averages-Based Optical and SAR Image Registration

Abstract

Optical and synthetic aperture radar (SAR) images exhibit non-negligible intensity differences due to their unique imaging mechanisms, which makes it difficult for classical SIFT-based algorithms to obtain sufficiently correct correspondences when processing the registration of these two types of images. To tackle this problem, an accurate optical and SAR image registration algorithm based on the SIFT algorithm (OS-PSO) is proposed. First, a modified ratio of exponentially weighted averages (MROEWA) operator is introduced to resolve the sudden dark patches in SAR images, thus generating more consistent gradients between optical and SAR images. Next, we innovatively construct the Harris scale space to replace the traditional difference in the Gaussian (DoG) scale space, identify repeatable key-points by searching for local maxima, and perform localization refinement on the identified key-points to improve their accuracy. Immediately after that, the gradient location orientation histogram (GLOH) method is adopted to construct the feature descriptors. Finally, we propose an enhanced matching method. The transformed relation is obtained in the initial matching stage using the nearest neighbor distance ratio (NNDR) and fast sample consensus (FSC) methods. And the re-matching takes into account the location, scale, and main direction of key-points to increase the number of correctly corresponding points. The proposed OS-PSO algorithm has been implemented on the Gaofen and Sentinel series with excellent results. The superior performance of the designed registration system can also be applied in complex scenarios, including urban, suburban, river, farmland, and lake areas, with more efficiency and accuracy than the state-of-the-art methods based on the WHU-OPT-SAR dataset and the BISTU-OPT-SAR dataset.

Keywords: MROEWA; SIFT algorithm; enhanced matching method; image registration.

Figure 1

Overall framework of our proposed OS-PSO algorithm.

Figure 2

Illustrations of gradient images with dark patches at the same scale. (a) SAR image with dark patches; (b) ROEWA-processed SAR image; (c) MROEWA-processed SAR image.

Figure 3

Impact of dark patches on gradient magnitudes at the same scale. (a) Sobel operator in the optical image; (b) ROEWA operator in the SAR image. (c) MROEWA operator in the SAR image.

Figure 4

Repeatability of key-points in the BISTU-OPT-SAR dataset and WHU-OPT-SAR dataset with different location errors. (a–e) represents location error t = {1, 2, 3, 4, 5}.

Figure 5

The performance of different key-point matching methods. (a) Average CMN; (b) average CMR, (c) average RMSE.

Figure 6

Registration results of the OS-PSO algorithm based on four image pairs with geometric differences. (a) P1 with significant rotation differences and slight scale differences; (b) P2 with significant scale differences; (c) P3 with slight scale and rotation differences; (d) P4 with significant translation and rotation differences.

Figure 7

Six image pairs from the BISTU-OPT-SAR and WHU-OPT-SAR datasets. (a–d) Pairs 1,2,3, and 4 from the BISTU-OPT-SAR dataset; (e,f) Pairs 5 and 6 are from the WHU-OPT-SAR dataset.

Figure 8

Registration results of different algorithms based on six image pairs from different scenarios, including urban, suburban, river, farmland, and lake. (a) PSO-SIFT; (b) SAR-SIFT; (c) OS-SIFT; (d) RIFT; (e) OSS; (f) MROEWA+FSC; (g) OS-SIFT+PSO; (h) OS-PSO.

Figure 9

Registration results of OS-SIFT+PSO, MROEWA+FSC, and the proposed OS-PSO algorithms based on six image pairs with different geographic environments. (a) Urban Area; (b) suburban area; (c) river area; (d) river area; (e) farmland area; (f) lake area.