Benchmarking Domain Adaptation Methods on Aerial Datasets

Abstract

Deep learning grew in importance in recent years due to its versatility and excellent performance on supervised classification tasks. A core assumption for such supervised approaches is that the training and testing data are drawn from the same underlying data distribution. This may not always be the case, and in such cases, the performance of the model is degraded. Domain adaptation aims to overcome the domain shift between the source domain used for training and the target domain data used for testing. Unsupervised domain adaptation deals with situations where the network is trained on labeled data from the source domain and unlabeled data from the target domain with the goal of performing well on the target domain data at the time of deployment. In this study, we overview seven state-of-the-art unsupervised domain adaptation models based on deep learning and benchmark their performance on three new domain adaptation datasets created from publicly available aerial datasets. We believe this is the first study on benchmarking domain adaptation methods for aerial data. In addition to reporting classification performance for the different domain adaptation models, we present t-SNE visualizations that illustrate the benefits of the adaptation process.

Keywords: aerial datasets; deep neural networks; domain adaptation; unsupervised learning; visualization.

Figure 1

(a) Closed set DA has same classes in source and target domains. (b) Both source and target domains in open set DA contain contain data that do not belong to classes of interest. Unknown samples are represented in green. Further, target domain can contain data from classes not related to classes in source domain.

Figure 2

Architecture of SymNets [53]. Blue and red arrows indicate source and target domains and losses corresponding to them respectively. Yellow refers to feature extractor and corresponding losses and green represents classifiers and their losses.

Figure 3

Architecture of RSDA [54]. Blue and red arrows represent computational flow of source and target domain samples respectively. F is a feature extractor which is a CNN that extracts features and embeds them onto a hypersphere. Spherical classifier predicts class labels and domain discriminator predicts domain labels. Posterior probability of correct labels is obtained by feeding the target pseudo-labels and target features into a Gaussian mixture model. Posterior probabilities then weight pseudo-label loss for robustness.

Figure 4

Network architecture of Source Hypothesis Transfer (SHOT) [57].

Figure 5

Sample images from shared classes between AID (top row) and UCM (bottom row). Classes are (from left to right) baseball field/baseball diamond, beach, medium residential, sparse residential, parking/parking lot, airport/airplane, storage tank, forest, and river.

Figure 6

Sample images from shared classes between NWPU (top row) and CLRS (bottom row). Classes are (from left to right) airplane, bridge, parking, railway station, runway, and storage tank.

Figure 7

Sample images from shared classes between DOTA (top row) and xView (bottom row). Classes are (from left to right) large vehicle, plane, ship, small vehicle, and storage tank.

Figure 8

SymNets results for xView-DOTA dataset, before adaptation (left) and after adaptation (right).

Figure 9

RSDA results for xView-DOTA dataset, before adaptation (left) and after adaptation (right).

Figure 10

CDAN-GD results for xView-DOTA dataset, before adaptation (left) and after adaptation (right).

Figure 11

GVB-GD results for xView-DOTA dataset, before adaptation (left) and after adaptation (right).

Figure 12

UAN results for xView-DOTA dataset, before adaptation (left) and after adaptation (right).

Figure 13

SHOT results for xView-DOTA dataset, before adaptation (left) and after adaptation (right).

Figure 14

SRDC results for xView-DOTA dataset, before adaptation (left) and after adaptation (right).

Figure 15

SymNets t-SNE for xView-DOTA dataset, before adaptation (left) and after adaptation (right). Blue points correspond to source domain (xView), and red points to target domain (DOTA).

Figure 16

RSDA t-SNE for xView-DOTA dataset, before adaptation (left) and after adaptation (right). Blue points correspond to source domain (xView), and red points to target domain (DOTA).

Figure 17

CDAN-GD t-SNE for xView-DOTA dataset, before adaptation (left) and after adaptation (right). Blue points correspond to source domain (xView), and red points to target domain (DOTA).

Figure 18

GVB-GD t-SNE for xView-DOTA dataset, before adaptation (left) and after adaptation (right). Blue points correspond to source domain (xView), and red points to target domain (DOTA).

Figure 19

UAN t-SNE for xView-DOTA dataset, before adaptation (left) and after adaptation (right). Blue points correspond to source domain (xView), and red points to target domain (DOTA).

Figure 20

SHOT t-SNE for xView-DOTA dataset, before adaptation (left) and after adaptation (right). Blue points correspond to source domain (xView), and red points to target domain (DOTA).

Figure 21

SRDC t-SNE for xView-DOTA dataset, before adaptation (left) and after adaptation (right). Blue points correspond to source domain (xView), and red points to target domain (DOTA).