Upcoming SlideShare
×

# Relative Importance Weight for Covariate Shift Adaptation

1,689 views

Published on

Published in: Technology, Health & Medicine
2 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

Views
Total views
1,689
On SlideShare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
8
0
Likes
2
Embeds 0
No embeds

No notes for slide

### Relative Importance Weight for Covariate Shift Adaptation

1. 1. 1Relative Importance Weight for Covariate Shift Adaptation Makoto Yamada Tokyo Institute of Technology April/21/2012 (Ver.0)
2. 2. Covariate Shift Adaptation (JSPI 2000) 2 ShimodairaTraining data:Test data :Assumption :Importance weighted empirical error minimization: We can obtain unbiased model in theory. But, it usually gives unsatisfactory results… Why?
3. 3. A Problem in Covariate Shift Adaptation 3Importance weight can diverge to infinity under a rather simple setting. Cortes et al. (NIPS 2010) In this situation, the covariate shift adaptation is unstable since estimated importance weight is unstable
4. 4. Exponentially-flattened IW (EIW) 4 empirical error minimization Shimodaira (JSPI 2000)Flatten the importance weight by  empirical error minimization.  Intermediate  IW empirical error minimization Setting to is practically useful for stabilizing the covariate shift adaptation, even though it cannot give an unbiased model under covariate shift. It still needs importance weight estimation 
5. 5. Relative importance-weighted (RIW) 5 empirical error minimizational. (NIPS 2011) Yamada etUse relative importance weight (RIW): If , RIW is bounded. Thus, estimating RIW is easier than estimating IW. RIW can be efficiently estimated by RuLSIF. http://sugiyama-www.cs.titech.ac.jp/~yamada/RuLSIF.htmlRIW-empirical error minimization: works well in practice.
6. 6. Toy Example 6Comparison EIW and RIW LS: least-squares regression RIW method gives smaller error and variance
7. 7. Real Experiments 7 (Human Activity Recognition)Data: Accelerometer data collected by iPod touchActivities: Walking, running, and bicycle ridingTraining data: 20 existing usersTest data: New usersClassifier: Kernel Logistic Regression (KLR) RIW method is also useful for practical data
8. 8. Summary 8Covariate shift adaptation tends to be unstable.Relative importance weight (RIW) is useful to stabilize the covariate shift adaptation. ( works well in practice)
9. 9. Useful Softwares 9RuLSIF http://sugiyama-www.cs.titech.ac.jp/~yamada/RuLSIF.htmlIWLS (Regression) http://sugiyama-www.cs.titech.ac.jp/~sugi/software/IWLS/index.htmlIWKLR (Classification) http://sugiyama-www.cs.titech.ac.jp/~yamada/iwklr.htmlIWLSPC (Classification) http://sugiyama-www.cs.titech.ac.jp/~hachiya/software/IWLSPC/index.html