Source-free unsupervised domain adaptation: A survey

Abstract

Unsupervised domain adaptation (UDA) via deep learning has attracted appealing attention for tackling domain-shift problems caused by distribution discrepancy across different domains. Existing UDA approaches highly depend on the accessibility of source domain data, which is usually limited in practical scenarios due to privacy protection, data storage and transmission cost, and computation burden. To tackle this issue, many source-free unsupervised domain adaptation (SFUDA) methods have been proposed recently, which perform knowledge transfer from a pre-trained source model to the unlabeled target domain with source data inaccessible. A comprehensive review of these works on SFUDA is of great significance. In this paper, we provide a timely and systematic literature review of existing SFUDA approaches from a technical perspective. Specifically, we categorize current SFUDA studies into two groups, i.e., white-box SFUDA and black-box SFUDA, and further divide them into finer subcategories based on different learning strategies they use. We also investigate the challenges of methods in each subcategory, discuss the advantages/disadvantages of white-box and black-box SFUDA methods, conclude the commonly used benchmark datasets, and summarize the popular techniques for improved generalizability of models learned without using source data. We finally discuss several promising future directions in this field.

Keywords: Domain adaptation; Source-free; Survey; Unsupervised learning.

Fig. 1.

Illustration of (a) conventional unsupervised domain adaptation (UDA), (b) white-box source-free UDA (SFUDA), and (c) black-box SFUDA. Compared with (a) conventional UDA that relies on labeled source data {X_S, Y_S} and unlabeled target data X_T, (b, c) SFUDA performs knowledge transfer by directly leveraging a pre-trained source model Φ_S and unlabeled target data X_T. The difference between (b) white-box SFUDA and (c) black-box SFUDA lies in whether the learnable parameters of the source model Φ_S are accessible or not. API: application programming interface.

Fig. 2.

Taxonomy of existing source-free unsupervised domain adaptation (SFUDA) methods and future outlook.

Fig. 3.

Overview of the number of SFUDA studies (in terms of published years) that have been collected In this survey.

Fig. 4.

Illustration of Batch Normalization Statistics Transfer methods for source Image generation. By matching batch normalization (BN) statistics between the upper and lower branches, source-like data can be generated by preserving the target content but with source style. Unsupervised domain adaptation (UDA) is then performed between source-like data and target data.

Fig. 5.

Illustration of Surrogate Source Data Construction methods for source data generation. These methods first construct surrogate/proxy source data by selecting appropriate samples from the target domain and then perform standard unsupervised domain adaptation (UDA).

Fig. 6.

Illustration of Generative Adversarial Network (GAN) based Image Generation methods for source data generation. Typically, a pre-defined label and random noise act as the Inputs of a GAN-based generator. By utilizing the pre-trained source model, they synthesize source data for cross-domain adaptation. L_CE: Cross-entropy loss function.

Fig. 7.

Illustration of Self-Supervised Knowledge Distillation methods for source-free unsupervised domain adaptation. With target data from different augmentations as Inputs, a teacher-student framework Is utilized to exploit target features, where parameters of teacher network are usually exponential moving average (EMA) of those of student network. Aug-α and Aug-β denote two data augmentation methods (e.g., flip, rotation, shift, noise addition, distortion, etc.), respectively. L_KD: Knowledge distillation loss function.

Fig. 8.

Illustration of Domain Alignment via Statistics methods for source-free unsupervised domain adaptation. The corresponding methods leverage batch statistics stored in the pre-trained source model to approximate the distribution of inaccessible source data, and then perform cross-domain adaptation by reducing distribution discrepancy between source and target domains.

Fig. 9.

Illustration of Contrastive Learning methods for source-free unsupervised domain adaptation. These methods exploit discriminative representations among unlabeled target data by pulling instances of similar categories closer and pushing instances of different categories away in feature space.

Fig. 10.

Illustration of Uncertainty-Guided Adaptation methods for source-free unsupervised domain adaptation. These studies utilize uncertainty to guide target predictions, and such valuable information can be measured by Monte Carlo Dropout, entropy, etc.

Fig. 11.

Illustration of Hidden Structure Mining methods for source-free unsupervised domain adaptation. These methods take into consideration intrinsic feature structures of the target domain and iterate between target model refinement and clustering centroid update.