Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
×

# Understanding Black-box Predictions via Influence Functions (2017)

30,683 views

Published on

"Understanding Black-box Predictions via Influence Functions" (2017),
PangWei Koh and Percy Liang,
ICML2017 Best paper

Published in: Engineering
• Full Name
Comment goes here.

Are you sure you want to Yes No
Your message goes here
• Be the first to comment

### Understanding Black-box Predictions via Influence Functions (2017)

1. 1. Terry Taewoong Um (terry.t.um@gmail.com) University of Waterloo Department of Electrical & Computer Engineering Terry T. Um UNDERSTANDING BLACK-BOX PRED -ICTION VIA INFLUENCE FUNCTIONS 1
2. 2. TODAY’S PAPER Terry Taewoong Um (terry.t.um@gmail.com) ICML2017 best paper https://youtu.be/0w9fLX_T6tY
3. 3. QUESTIONS Terry Taewoong Um (terry.t.um@gmail.com) • How can we explain the predictions of a black-box model? • Why did the system make this prediction? • How can we explain where the model came from? • What would happen if the values of a training point where slightly changed?
4. 4. INTERPRETATION OF DL RESULTS Terry Taewoong Um (terry.t.um@gmail.com) • Retrieving images that maximally activate a neuron [Girshick et al. 2014] • Finding the most influential part from the image [Zhou et al. 2016] • Learning a simpler model around a test point [Ribeiro et al. 2016] But, they assumed a fixed model  My NN is a function of training inputs
5. 5. INFLUENCE OF A TRAINING POINT Terry Taewoong Um (terry.t.um@gmail.com) • What is the influence of a training example for the model (or for the loss of a test example)? Optimal model param. : Model param. by training w/o z : Model param. by upweighting z : without z == (𝜖 = − 1 𝑛 ) • The influence of upweighting z on the parameters 𝜃
6. 6. INFLUENCE OF A TRAINING POINT • Influence vs. Euclidean distance
7. 7. INFLUENCE OF A TRAINING POINT Terry Taewoong Um (terry.t.um@gmail.com) • The influence of upweighting z on the parameters 𝜃 • The influence of upweighting z on the loss at a test point
8. 8. PERTURBING A TRAINING POINT Terry Taewoong Um (terry.t.um@gmail.com) • Move 𝜖 mass from 𝑧 to 𝑧 𝛿 • If x is continuous and 𝛿 is small • The effect of 𝑧  𝑧 𝛿 on the loss at a test point
9. 9. SUMMARY Terry Taewoong Um (terry.t.um@gmail.com) • The influence of 𝑧  𝑧 𝛿 on the loss at a test point • The influence of upweighting z on the parameters 𝜃 • The influence of upweighting z on the loss at a test point
10. 10. EXAMPLE Terry Taewoong Um (terry.t.um@gmail.com) • The influence of upweighting z • In logistic regression, • Test : 7, Train : 7 (green), 1 (red)
11. 11. SEVERAL PROBLEMS Terry Taewoong Um (terry.t.um@gmail.com) • Calculation of  Use Hessian-vector products (HVPs)  precompute 𝑠𝑡𝑒𝑠𝑡 by optimizing or sampling-based approximation
12. 12. SEVERAL PROBLEMS Terry Taewoong Um (terry.t.um@gmail.com) • What if is non-convex, so H < 0  Assuming that is a local minimum point, define a quadratic loss Then calculate using the above  empirically working! • Influence function vs. retraining
13. 13. SEVERAL PROBLEMS Terry Taewoong Um (terry.t.um@gmail.com) • What if is non-differentiable? e.g.) hinge loss  Use a differentiable variation of the hinge loss
14. 14. APPLICATIONS Terry Taewoong Um (terry.t.um@gmail.com) • Understanding model behavior
15. 15. APPLICATIONS Terry Taewoong Um (terry.t.um@gmail.com) • Adversarial examples c.f.) The effect of 𝑧  𝑧 𝛿 on the loss at a test point
16. 16. APPLICATIONS Terry Taewoong Um (terry.t.um@gmail.com) • Debugging domain mismatch
17. 17. APPLICATIONS Terry Taewoong Um (terry.t.um@gmail.com) • Fixing mislabeled examples