Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Successfully reported this slideshow.

Like this presentation? Why not share!

- Deformable Convolutional Network (2... by Terry Taewoong Um 33803 views
- Learning with side information thro... by Terry Taewoong Um 28728 views
- Human Motion Forecasting (Generatio... by Terry Taewoong Um 30466 views
- Introduction to Machine Learning an... by Terry Taewoong Um 61316 views
- 기계학습(Machine learning) 입문하기 by Terry Taewoong Um 67072 views
- Lie Group Formulation for Robot Mec... by Terry Taewoong Um 14666 views

26,969 views

Published on

PangWei Koh and Percy Liang,

ICML2017 Best paper

Published in:
Engineering

License: CC Attribution-NonCommercial License

No Downloads

Total views

26,969

On SlideShare

0

From Embeds

0

Number of Embeds

24,481

Shares

0

Downloads

151

Comments

0

Likes

3

No embeds

No notes for slide

- 1. Terry Taewoong Um (terry.t.um@gmail.com) University of Waterloo Department of Electrical & Computer Engineering Terry T. Um UNDERSTANDING BLACK-BOX PRED -ICTION VIA INFLUENCE FUNCTIONS 1
- 2. TODAY’S PAPER Terry Taewoong Um (terry.t.um@gmail.com) ICML2017 best paper https://youtu.be/0w9fLX_T6tY
- 3. QUESTIONS Terry Taewoong Um (terry.t.um@gmail.com) • How can we explain the predictions of a black-box model? • Why did the system make this prediction? • How can we explain where the model came from? • What would happen if the values of a training point where slightly changed?
- 4. INTERPRETATION OF DL RESULTS Terry Taewoong Um (terry.t.um@gmail.com) • Retrieving images that maximally activate a neuron [Girshick et al. 2014] • Finding the most influential part from the image [Zhou et al. 2016] • Learning a simpler model around a test point [Ribeiro et al. 2016] But, they assumed a fixed model My NN is a function of training inputs
- 5. INFLUENCE OF A TRAINING POINT Terry Taewoong Um (terry.t.um@gmail.com) • What is the influence of a training example for the model (or for the loss of a test example)? Optimal model param. : Model param. by training w/o z : Model param. by upweighting z : without z == (𝜖 = − 1 𝑛 ) • The influence of upweighting z on the parameters 𝜃
- 6. INFLUENCE OF A TRAINING POINT • Influence vs. Euclidean distance
- 7. INFLUENCE OF A TRAINING POINT Terry Taewoong Um (terry.t.um@gmail.com) • The influence of upweighting z on the parameters 𝜃 • The influence of upweighting z on the loss at a test point
- 8. PERTURBING A TRAINING POINT Terry Taewoong Um (terry.t.um@gmail.com) • Move 𝜖 mass from 𝑧 to 𝑧 𝛿 • If x is continuous and 𝛿 is small • The effect of 𝑧 𝑧 𝛿 on the loss at a test point
- 9. SUMMARY Terry Taewoong Um (terry.t.um@gmail.com) • The influence of 𝑧 𝑧 𝛿 on the loss at a test point • The influence of upweighting z on the parameters 𝜃 • The influence of upweighting z on the loss at a test point
- 10. EXAMPLE Terry Taewoong Um (terry.t.um@gmail.com) • The influence of upweighting z • In logistic regression, • Test : 7, Train : 7 (green), 1 (red)
- 11. SEVERAL PROBLEMS Terry Taewoong Um (terry.t.um@gmail.com) • Calculation of Use Hessian-vector products (HVPs) precompute 𝑠𝑡𝑒𝑠𝑡 by optimizing or sampling-based approximation
- 12. SEVERAL PROBLEMS Terry Taewoong Um (terry.t.um@gmail.com) • What if is non-convex, so H < 0 Assuming that is a local minimum point, define a quadratic loss Then calculate using the above empirically working! • Influence function vs. retraining
- 13. SEVERAL PROBLEMS Terry Taewoong Um (terry.t.um@gmail.com) • What if is non-differentiable? e.g.) hinge loss Use a differentiable variation of the hinge loss
- 14. APPLICATIONS Terry Taewoong Um (terry.t.um@gmail.com) • Understanding model behavior
- 15. APPLICATIONS Terry Taewoong Um (terry.t.um@gmail.com) • Adversarial examples c.f.) The effect of 𝑧 𝑧 𝛿 on the loss at a test point
- 16. APPLICATIONS Terry Taewoong Um (terry.t.um@gmail.com) • Debugging domain mismatch
- 17. APPLICATIONS Terry Taewoong Um (terry.t.um@gmail.com) • Fixing mislabeled examples

No public clipboards found for this slide

Be the first to comment