Understanding Blackbox Prediction via Influence Functions

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Introduction
Introduction

Introduction
A key question often asked of machine learning systems is
“Why did the system make this prediction?”
How can ...

Introduction
Answering this question by perturbing the data and retraining the model
can be prohibitively expensive. To ov...

Methods
Methods

Approach
We are given training points 𝑧1,… , 𝑧 𝑛, where 𝑧𝑖 = (𝑥𝑖, 𝑦𝑖) ∈ X × Y. For
a point 𝑧 and parameters 𝜃 ∈ Θ, let 𝐿(𝑧...

Approach
Model param. by training w/o z :
Model param. by upweighting z :
Model param. by perturbing z :

Approach
Let us begin by studying the change in model parameters due to
removing a point z from the training set.
Formally...

Influence function - proof of up,params

Influence function - proof of up,params

Up, params influence
where 𝐻෡𝜃 ≝
1
𝑛
σ𝑖=1
𝑛
∇ 𝜃
2
𝐿(𝑧, ෠𝜃) is the Hessian and is positive deﬁnite
(PD) by assumption. In e...

influence loss of up, params

Perturbing a training input
For a training point 𝑧 = (𝑥, 𝑦) , define 𝑧 𝛿 ≝ (𝑥 + 𝛿, 𝑦). Consider the
perturbation 𝑧 → 𝑧 𝛿 ,...

Perturbing a training input
If x is continuous and 𝛿is small
lim
ℎ→0
F(X+h) – F(X) = F’(X)∗h

Efficiently calculation
How to calculate it?

Efficiently calculation
We discuss two techniques for approximating 𝑠𝑡𝑒𝑠𝑡, both relying on
the fact that the HVP of a sing...

Efficiently calculation - Conjugate gradients (CG)
Since 𝐻෡𝜃 ≻ 0 by assumption, 𝐻෡𝜃
−1
𝑣 ≡ 𝑎𝑟𝑔𝑚𝑖𝑛 𝑡
1
2
𝑡 𝑇 𝐻෡𝜃 𝑡 − 𝑣 𝑇
𝑡 ...

Efficiently calculation - Stochastic estimation
𝑠𝑡𝑒𝑠𝑡 ≝ 𝐻෡𝜃
−1
∇ 𝜃 𝐿(𝑧𝑡𝑒𝑠𝑡, ෠𝜃)
Dropping the ෠𝜃 subscript for clarity,let ...

Efficiently calculation - Stochastic estimation
෪𝐻𝑗
−1
𝑣 = 𝑣 + (𝐼 − ∇ 𝜃
2
𝐿(𝑧𝑠 𝑗
, ෠𝜃))෫𝐻𝑗−1
−1
𝑣
Empirically, we found th...

Non-convexity and non-convergence
Our approach is to form a convex quadratic approximation of the loss
around ෩𝜃 , i.e., ෩...

IHVP by Lissa Algorithms

Applications

Applications

Applications - Understanding model behavior
Inﬂuence functions reveal insights about
how models rely on and extrapolate fr...

Applications - Understanding model behavior

Applications

Application - Adversarial training examples
Training datasets are vulnerable to attack
Can we create adversarial training ...

Applications

Application - Debugging domain mismatch
If a model makes a mistake, can we find out why?
Original Modified
~20k -> ~20k
21...

Application - Debugging domain mismatch
True test label: Healthy children
Model predicts: Re-admitted childeren
0.1
0
-0.1...

Applications

Application - Fixing mislabeled examples
Training labels are noisy, and we have a small budget to manually inspect them
Ca...

$Application - Fixing mislabeled examples Plots of how test accuracy (left) and the fraction of ﬂipped data detected (right...$

References
References

References
Pang Wei Koh and Percy Liang. "Understanding Black-Box prediction via influence functions" ICML 2017 Best
paper...

Understanding Blackbox Prediction via Influence Functions