As Data Scientists we want to understand machine learning models we have built. “Why did my model make this mistake?”, “Does my model discriminate?”, “How can I understand and trust the model's decisions?”, “Does my model satisfy legal requirements?” are commonly asked questions. In this presentation we will talk about machine learning explainability and interpretability - two concepts that could help us really understand ML models. Website: https://fwdays.com/en/event/data-science-fwdays-2019/review/explaining-a-machine-learning-blackbox

1. Explaining a Machine Learning
blackbox
Oleksandr Krakovetskyi, Ph.D.
DevRain

2. About

3. What is interpretability?
There is no mathematical definition of interpretability.
A (non-mathematical) definition by Miller (2017) is:
Interpretability is the degree to which a human can understand the
cause of a decision.
Another one is: Interpretability is the degree to which a human can
consistently predict the model’s result.

4. Interpretability is essential for:
1. Model debugging - Why did my model make this mistake?
2. Detecting bias - Does my model discriminate?
3. Human-AI cooperation - How can I understand and trust the
model's decisions?
4. Regulatory compliance - Does my model satisfy legal
requirements?
5. High-risk applications - Healthcare, finance, judicial, ...

5. Accuracy vs. Interpretability

6. Text Classification

7. Loan Approval
Me: I want a loan. Can I?
Bank: No.

8. Husky or Malamute?

9. Why it’s important?
Interpretability is a key to winning
trust in algorithms that try to
improve upon human judgement,
instead of just automating it.
What is the Answer to the Ultimate
Question of Life, The Universe,
and Everything? Answer is:
42

10. Why it’s important? Bias and Ethics.

11. Why it’s important? Regulations.
GDPR
General Data Protection Regulation
Right to explanation is a demand from the European Union to
make Artificial Intelligence more transparent and ethical.
This regulation promotes to build Algorithms that ensures an
explanation for every Machine Learning decision.

12. Dimensions of Interpretability
1. Scope: Global vs. Local
2. When to explain: Before, During, After model’s creation
3. Model type: Specific vs. Agnostic
4. Recipient of explanation: User, Manager, Data Scientist

13. Interpretability before model creation
1. Exploratory data analysis
2. Understand distributions
3. Simple feature analysis
4. Clustering

14. Interpretability during model creation
1. Fewer features
2. Linear models
3. Decision trees

15. Linear models

16. Decision trees

17. Interpretability after model creation
1. Explain the model
2. Explain the outcome
3. Inspect the black box internally

18. Tools and Frameworks
1. LIME https://github.com/marcotcr/lime
2. Lucid https://github.com/tensorflow/lucid
3. Shap https://github.com/slundberg/shap
4. RNNVis https://github.com/myaooo/RNNVis
5. Keras-Vis https://github.com/raghakot/keras-vis
6. What-if by Google https://pair-code.github.io/what-if-tool/
7. Fairness 360 Kit by IBM https://github.com/IBM/aif360
8. InterpretML by Microsoft https://github.com/Microsoft/interpret

19. LIME. Two-class case

20. LIME. Multi-class case

21. LIME. Tabular data

22. LIME. Image classification

23. Explaining a prediction with LIME

24. Model interpretability with Azure
Machine Learning service

25. Blackbox interpretability for binary
classification with InterpretML
Demo
https://nbviewer.jupyter.org/github/Microsoft/interpret/blob
/master/examples/python/notebooks/Explaining%20Blackbox
%20Classifiers.ipynb

26. Thanks!