1Relative Importance Weight for  Covariate Shift Adaptation          Makoto Yamada    Tokyo Institute of Technology       ...
Covariate Shift Adaptation (JSPI 2000)    2                              ShimodairaTraining data:Test data    :Assumpti...
A Problem in Covariate Shift Adaptation 3Importance weight can diverge to infinity under a rather simple setting.        ...
Exponentially-flattened IW (EIW)                              4       empirical error minimization                        ...
Relative importance-weighted (RIW) 5       empirical error minimizational. (NIPS 2011)                              Yamada...
Toy Example                     6Comparison EIW and RIW  LS: least-squares regression  RIW method gives smaller error and...
Real Experiments                         7      (Human Activity Recognition)Data: Accelerometer data collected by iPod to...
Summary                         8Covariate shift adaptation tends to be unstable.Relative importance weight (RIW) is use...
Useful Softwares                                          9RuLSIF http://sugiyama-www.cs.titech.ac.jp/~yamada/RuLSIF.html...
Upcoming SlideShare
Loading in …5
×

Relative Importance Weight for Covariate Shift Adaptation

1,689 views

Published on

Published in: Technology, Health & Medicine
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,689
On SlideShare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
8
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Relative Importance Weight for Covariate Shift Adaptation

  1. 1. 1Relative Importance Weight for Covariate Shift Adaptation Makoto Yamada Tokyo Institute of Technology April/21/2012 (Ver.0)
  2. 2. Covariate Shift Adaptation (JSPI 2000) 2 ShimodairaTraining data:Test data :Assumption :Importance weighted empirical error minimization: We can obtain unbiased model in theory. But, it usually gives unsatisfactory results… Why?
  3. 3. A Problem in Covariate Shift Adaptation 3Importance weight can diverge to infinity under a rather simple setting. Cortes et al. (NIPS 2010) In this situation, the covariate shift adaptation is unstable since estimated importance weight is unstable
  4. 4. Exponentially-flattened IW (EIW) 4 empirical error minimization Shimodaira (JSPI 2000)Flatten the importance weight by  empirical error minimization.  Intermediate  IW empirical error minimization Setting to is practically useful for stabilizing the covariate shift adaptation, even though it cannot give an unbiased model under covariate shift. It still needs importance weight estimation 
  5. 5. Relative importance-weighted (RIW) 5 empirical error minimizational. (NIPS 2011) Yamada etUse relative importance weight (RIW): If , RIW is bounded. Thus, estimating RIW is easier than estimating IW. RIW can be efficiently estimated by RuLSIF. http://sugiyama-www.cs.titech.ac.jp/~yamada/RuLSIF.htmlRIW-empirical error minimization: works well in practice.
  6. 6. Toy Example 6Comparison EIW and RIW LS: least-squares regression RIW method gives smaller error and variance
  7. 7. Real Experiments 7 (Human Activity Recognition)Data: Accelerometer data collected by iPod touchActivities: Walking, running, and bicycle ridingTraining data: 20 existing usersTest data: New usersClassifier: Kernel Logistic Regression (KLR) RIW method is also useful for practical data
  8. 8. Summary 8Covariate shift adaptation tends to be unstable.Relative importance weight (RIW) is useful to stabilize the covariate shift adaptation. ( works well in practice)
  9. 9. Useful Softwares 9RuLSIF http://sugiyama-www.cs.titech.ac.jp/~yamada/RuLSIF.htmlIWLS (Regression) http://sugiyama-www.cs.titech.ac.jp/~sugi/software/IWLS/index.htmlIWKLR (Classification) http://sugiyama-www.cs.titech.ac.jp/~yamada/iwklr.htmlIWLSPC (Classification) http://sugiyama-www.cs.titech.ac.jp/~hachiya/software/IWLSPC/index.html

×