7. Algorithms decide which emails
reach our inboxes, whether we are
approved for credit, and whom we
get the opportunity to date.
But, as algorithms give answers,
they raise questions. If an algorithm
denies your loan application,
wouldn’t you like to know why or
what you could change for a more
positive outcome?
Or perhaps you’d like to know if
your bank is right to trust the
algorithm in the first place.
8.
9. We need models that we know are right for the
right reasons and wrong for the right reasons.
10. Traditional model
validation only tells us
how well we do on the
holdout validation set.
It does not tell us whether
we are right for the right
reasons and wrong for
the right reasons.
11. This model is “right”
(i.e., scores high on
the holdout
validation set) for
the wrong reasons.
12. Knowing the reasons for a
prediction may also provide
us with strategies to change
the predicted outcome.
13. The Benefits of Interpretability
• Verify the model is right for the right reasons and wrong for the
right reasons (and develop trust in the model).
• Explain individual model decisions and guide intervention.
• Satisfy regulators and ethical concerns (in EU: GDPR in May 2018).
• Develop a more collaborative relationship with models (AI)
44. Data are traces of past activity.
Behavioral data encode past behavioral patterns,
some of which reflect outdated conventions,
prejudice, or harmful past practices.
47. The Benefits of Interpretability
• Verify the model is right for the right reasons and wrong for the
right reasons (and develop trust in the model).
• Explain individual model decisions and guide intervention.
• Satisfy regulators and ethical concerns (in EU: GDPR in May 2018).
• Develop a more collaborative relationship with models (AI)
48. The Benefits of LIME
• Model-agnostic interpretability solutions like LIME avoid the
interpretability-accuracy trade off.
• Model-agnostic interpretability solutions like LIME work with any model,
even the ones already trained.
• LIME is easy to use, it can extend any machine learning pipeline.
• LIME works best with tabular data (continuous and categorical).
49. Resources
• Blog post (http://blog.fastforwardlabs.com/2017/09/01/LIME-for-couples.html) and
notebook (https://github.com/fastforwardlabs/couples-lime/blob/master/couples-lime.ipynb)
• LIME paper (https://arxiv.org/abs/1602.04938) and code (https://github.com/marcotcr/lime).
• Machine bias and recidivism (https://www.propublica.org/article/machine-bias-risk-
assessments-in-criminal-sentencing).
• Interpretability via influence functions (https://arxiv.org/abs/1703.04730).
• Attention layers in neural networks (e.g., http://www.cs.cmu.edu/~./hovy/papers/16HLT-
hierarchical-attention-networks.pdf).
• Research and conference on Fairness, Accountability, and Transparency in ML (https://
www.fatml.org/).