1. DECIDING IN THE DARK:A Survey of The Current
Ecosystem of Explainability
Techniques
By Coco Sack 1

2. WHO AM I?
 AI2 Incubator Intern
 Yale freshman (Computer Science & Psychology major)
 Child of AI Boom
 As of recent, an “Explainability Expert”
2

3. 3

4. WHO CARES ABOUT EXPLAINABILITY?
DATA-SCIENTISTS
To debug...
POLITICIANS
GDPR:
“a right to
explanation”
To protect the public...
EXECUTIVES
76%
said lack of
transparency
was seriously
impeding
adoption
To sell...
CONSUMERS
To trust...
4

5. EXPLAINABILITY: A HOT TOPIC
5

6. ???
6

7. 77

8. 88

9. DATA ANALYSIS
◂ Data must be complete, unique, credible, accurate, consistent, and
unbiased
◂ Errors or duplicates can undermine the model’s performance
◂ Outdated bias in the training data can re-entrench discrimination.
You are saying, 'Here's the data, figure out what the behavior is.'
That is an inherently fuzzy and statistical approach. The real
challenge of deep learning is that it's not modeling, necessarily, the
world around it. It's modeling the data it's getting. And that modeling
often includes bias and problematic correlations.”
-- Sheldon Fernandez, CEO of DarwinAI
9

10. INTRINSICALLY INTERPRETABLE MODELS
1. Regression 2. Additive 3. Tree Graphs 4. Decision Rules
10

11. OUTPUT ANALYSIS TECHNIQUES
11

12. WHAT KIND OF MODELS DO THEY WORK ON?
12

13. WHAT KIND OF MODELS DO THEY WORK ON?
13

14. WHAT DOES IT OUTPUT?
14

15. WHAT DOES IT OUTPUT?
15

16. LOCAL VS. GLOBAL?
16

17. OTHER EXPLAINABILITY RESOURCES
General Surveys of Techniques:
A Survey of Methods For Explaining
Black Box Models (2018)
Visual Analytics in Deep Learning
(2018)
Explaining Explanations (2019)
Peeking Inside the Black Box (2018)
Books and Presentations:
Interpretable Machine Learning (by
Christopher Molnar, 2019)
XAI (by Dave Gunning, 2017)
Explaining Explanations (2019)
Unique Technical Papers:
Generative Synthesis (Wong et al.)
Golden Eye++ (Hendricks et al.)
Grad-CAM (Selvaraju et al.)
DeepLift (Shrikumar et al.)
T-CAV (Kim et al.)
Ethical/Political Reports:
“Computer Says No” (by Ian Sample)
“Why We Need to Open the Black Box”
(by AJ Abdallat)
“The Importance of Interpretable
Machine learning” (by DJ Sakar)
17