• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Relative Importance Weight for Covariate Shift Adaptation

Relative Importance Weight for Covariate Shift Adaptation






Total Views
Views on SlideShare
Embed Views



2 Embeds 38

https://twitter.com 34
https://si0.twimg.com 4


Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Relative Importance Weight for Covariate Shift Adaptation Relative Importance Weight for Covariate Shift Adaptation Presentation Transcript

    • 1Relative Importance Weight for Covariate Shift Adaptation Makoto Yamada Tokyo Institute of Technology April/21/2012 (Ver.0)
    • Covariate Shift Adaptation (JSPI 2000) 2 ShimodairaTraining data:Test data :Assumption :Importance weighted empirical error minimization: We can obtain unbiased model in theory. But, it usually gives unsatisfactory results… Why?
    • A Problem in Covariate Shift Adaptation 3Importance weight can diverge to infinity under a rather simple setting. Cortes et al. (NIPS 2010) In this situation, the covariate shift adaptation is unstable since estimated importance weight is unstable
    • Exponentially-flattened IW (EIW) 4 empirical error minimization Shimodaira (JSPI 2000)Flatten the importance weight by  empirical error minimization.  Intermediate  IW empirical error minimization Setting to is practically useful for stabilizing the covariate shift adaptation, even though it cannot give an unbiased model under covariate shift. It still needs importance weight estimation 
    • Relative importance-weighted (RIW) 5 empirical error minimizational. (NIPS 2011) Yamada etUse relative importance weight (RIW): If , RIW is bounded. Thus, estimating RIW is easier than estimating IW. RIW can be efficiently estimated by RuLSIF. http://sugiyama-www.cs.titech.ac.jp/~yamada/RuLSIF.htmlRIW-empirical error minimization: works well in practice.
    • Toy Example 6Comparison EIW and RIW LS: least-squares regression RIW method gives smaller error and variance
    • Real Experiments 7 (Human Activity Recognition)Data: Accelerometer data collected by iPod touchActivities: Walking, running, and bicycle ridingTraining data: 20 existing usersTest data: New usersClassifier: Kernel Logistic Regression (KLR) RIW method is also useful for practical data
    • Summary 8Covariate shift adaptation tends to be unstable.Relative importance weight (RIW) is useful to stabilize the covariate shift adaptation. ( works well in practice)
    • Useful Softwares 9RuLSIF http://sugiyama-www.cs.titech.ac.jp/~yamada/RuLSIF.htmlIWLS (Regression) http://sugiyama-www.cs.titech.ac.jp/~sugi/software/IWLS/index.htmlIWKLR (Classification) http://sugiyama-www.cs.titech.ac.jp/~yamada/iwklr.htmlIWLSPC (Classification) http://sugiyama-www.cs.titech.ac.jp/~hachiya/software/IWLSPC/index.html