PyData NYC 2017 Friederike Schuur

1. Friederike Schüür
Research Engineer
@FSchueuer
fastforwardlabs.com
contact@fastforwardlabs.com
@FastForwardLabs

2. http://blog.fastforwardlabs.com/2017/08/02/interpretability.html

3. http://blog.fastforwardlabs.com/2017/09/01/LIME-for-couples.html

4. Why your relationship is likely to last, or not?

5. Why your relationship is likely to last, or not?
we predict

6. https://www.newyorker.com/cartoon/a19697

7. Algorithms decide which emails
reach our inboxes, whether we are
approved for credit, and whom we
get the opportunity to date.
But, as algorithms give answers,
they raise questions. If an algorithm
denies your loan application,
wouldn’t you like to know why or
what you could change for a more
positive outcome?
Or perhaps you’d like to know if
your bank is right to trust the
algorithm in the first place.

9. We need models that we know are right for the
right reasons and wrong for the right reasons.

10. Traditional model
validation only tells us
how well we do on the
holdout validation set.
It does not tell us whether
we are right for the right
reasons and wrong for
the right reasons.

11. This model is “right”
(i.e., scores high on
the holdout
validation set) for
the wrong reasons.

12. Knowing the reasons for a
prediction may also provide
us with strategies to change
the predicted outcome.

13. The Benefits of Interpretability
• Verify the model is right for the right reasons and wrong for the
right reasons (and develop trust in the model).
• Explain individual model decisions and guide intervention.
• Satisfy regulators and ethical concerns (in EU: GDPR in May 2018).
• Develop a more collaborative relationship with models (AI)

15. Accuracy -
Interpretability tradeoff:
accurate models tend to
be less interpretable.

16. Accuracy -
Interpretability tradeoff:
accurate models tend to
be less interpretable.

17. https://arxiv.org/abs/1602.04938 https://github.com/marcotcr/lime
Model-agnostic
interpretability: Local
Interpretable Model
Explanations (LIME).

18. https://arxiv.org/abs/1602.04938 https://github.com/marcotcr/lime

19. https://arxiv.org/abs/1602.04938 https://github.com/marcotcr/lime

20. https://arxiv.org/abs/1602.04938 https://github.com/marcotcr/lime

21. https://arxiv.org/abs/1602.04938 https://github.com/marcotcr/lime

22. https://arxiv.org/abs/1602.04938 https://github.com/marcotcr/lime

23. https://arxiv.org/abs/1602.04938 https://github.com/marcotcr/lime

24. https://arxiv.org/abs/1602.04938 https://github.com/marcotcr/lime

26. http://blog.fastforwardlabs.com/2017/09/01/LIME-for-couples.html

27. https://data.stanford.edu/hcmst

28. https://github.com/fastforwardlabs/couples-lime/blob/master/couples-lime.ipynb
Instantiate the LIME explainer object:

29. https://github.com/fastforwardlabs/couples-lime/blob/master/couples-lime.ipynb
Explain an individual prediction:

30. https://github.com/fastforwardlabs/couples-lime/blob/master/couples-lime.ipynb
Get an explanation (alongside prediction):

31. Let’s take a closer look:
https://github.com/fastforwardlabs/couples-lime/blob/master/couples-lime.ipynb

32. https://github.com/fastforwardlabs/couples-lime/blob/master/couples-lime.ipynb
Let’s construct a data preprocessing pipeline:

33. https://github.com/fastforwardlabs/couples-lime/blob/master/couples-lime.ipynb
And then train the model:

34. Interpretable feature as inputs to pipeline:
https://github.com/fastforwardlabs/couples-lime/blob/master/couples-lime.ipynb

35. https://github.com/fastforwardlabs/couples-lime/blob/master/couples-lime.ipynb

36. https://github.com/fastforwardlabs/couples-lime/blob/master/couples-lime.ipynb

37. https://github.com/fastforwardlabs/couples-lime/blob/master/couples-lime.ipynb

38. Can I use LIME for (algorithmic)
relationship advice?

39. https://github.com/fastforwardlabs/couples-lime/blob/master/couples-lime.ipynb
Currently, we’re at 0.73 chance:

40. https://github.com/fastforwardlabs/couples-lime/blob/master/couples-lime.ipynb
But hey, I just got a raise!

41. https://github.com/fastforwardlabs/couples-lime/blob/master/couples-lime.ipynb
What if we bought our apartment?

42. https://github.com/fastforwardlabs/couples-lime/blob/master/couples-lime.ipynb
Really, we should just get married:

44. Data are traces of past activity.
Behavioral data encode past behavioral patterns,
some of which reflect outdated conventions,
prejudice, or harmful past practices.

45. https://github.com/fastforwardlabs/couples-lime/blob/master/couples-lime.ipynb

46. https://www.propublica.org/article/machine-bias-risk-
assessments-in-criminal-sentencing

47. The Benefits of Interpretability
• Verify the model is right for the right reasons and wrong for the
right reasons (and develop trust in the model).
• Explain individual model decisions and guide intervention.
• Satisfy regulators and ethical concerns (in EU: GDPR in May 2018).
• Develop a more collaborative relationship with models (AI)

48. The Benefits of LIME
• Model-agnostic interpretability solutions like LIME avoid the
interpretability-accuracy trade off.
• Model-agnostic interpretability solutions like LIME work with any model,
even the ones already trained.
• LIME is easy to use, it can extend any machine learning pipeline.
• LIME works best with tabular data (continuous and categorical).

49. Resources
• Blog post (http://blog.fastforwardlabs.com/2017/09/01/LIME-for-couples.html) and
notebook (https://github.com/fastforwardlabs/couples-lime/blob/master/couples-lime.ipynb)
• LIME paper (https://arxiv.org/abs/1602.04938) and code (https://github.com/marcotcr/lime).
• Machine bias and recidivism (https://www.propublica.org/article/machine-bias-risk-
assessments-in-criminal-sentencing).
• Interpretability via influence functions (https://arxiv.org/abs/1703.04730).
• Attention layers in neural networks (e.g., http://www.cs.cmu.edu/~./hovy/papers/16HLT-
hierarchical-attention-networks.pdf).
• Research and conference on Fairness, Accountability, and Transparency in ML (https://
www.fatml.org/).