Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Apr:89:3449-3458.

Low-Dimensional Density Ratio Estimation for Covariate Shift Correction

Petar Stojanov  1 , Mingming Gong  2 , Jaime G Carbonell  3 , Kun Zhang  4
Affiliations

Low-Dimensional Density Ratio Estimation for Covariate Shift Correction

Petar Stojanov et al. Proc Mach Learn Res. 2019 Apr.

Abstract

Covariate shift is a prevalent setting for supervised learning in the wild when the training and test data are drawn from different time periods, different but related domains, or via different sampling strategies. This paper addresses a transfer learning setting, with covariate shift between source and target domains. Most existing methods for correcting covariate shift exploit density ratios of the features to reweight the source-domain data, and when the features are high-dimensional, the estimated density ratios may suffer large estimation variances, leading to poor prediction performance. In this work, we investigate the dependence of covariate shift correction performance on the dimensionality of the features, and propose a correction method that finds a low-dimensional representation of the features, which takes into account feature relevant to the target Y, and exploits the density ratio of this representation for importance reweighting. We discuss the factors affecting the performance of our method and demonstrate its capabilities on both pseudo-real and real-world data.

PubMed Disclaimer

Figures

Figure 1:
Figure 1:
(a): Estimation L2 error of β in Gaussian mixture toy dataset (b): Term 1 of bound. (c): Classification accuracy.
See this image and copyright information in PMC

References

    1. Storkey Amos. When training and test sets are different: characterizing learning transfer Dataset shift in machine learning, pages 3–28, 2009.
    1. Zadrozny Bianca. Learning and evaluating classifiers under sample selection bias In Proceedings of the twenty-first international conference on Machine learning, page 114 ACM, 2004.
    1. Heckman James J. Sample selection bias as a specification error (with an application to the estimation of labor supply functions), 1977.
    1. Sugiyama Masashi, Nakajima Shinichi, Kashima Hisashi, Buenau Paul V, and Kawanabe Motoaki. Direct importance estimation with model selection and its application to covariate shift adaptation In Advances in neural information processing systems, pages 1433–1440, 2008.
    1. Gretton Arthur, Smola Alexander J, Huang Jiayuan, Schmittfull Marcel, Borgwardt Karsten M, and Schölkopf Bernhard. Covariate shift by kernel mean matching. 2009.

LinkOut - more resources