Explainable AI for Ordinary Humans - Teresa Kubacka

www.teresa-kubacka.com
World Usability Day 2020 Switzerland – Human-Centered AI – 12.11.2020
Explainable AI
for ordinary humans
Teresa Kubacka, PhD

Translating
between humans
and computers
What will this talk be about?
Differential equations
Philosophy of trust
Terminator
Comprehensive,
end-to-end overview
of a domain given by
a Nobel laureate
How to save the world
Cool charts
Looking into
the kitchen
Inspiring
examples Mess
People
Becoming a
bit nerdy
Symbiosis

During the talk we will...
- Explore the problem: what is Explainable AI (XAI)?
- Build our own AI
- Explore the symbiosis between data work and UX design
- Look at some practical strategies around explainability

What is XAI?
It depends!
(It’s a mess…)
(...but a beautiful and exciting mess!)

What do people think?
Explainable is a
synonym to...
Interpretable
Reproducible
Responsible
Tested
Fair
Auditable
Accountable
Certifiable
Inspectable
Justifiable
Operational
Ready-to-Use
Reliable
Repeatable
Self-service
Transparent
Trusted
Unbiased
Understandable
Verifiable

What do people need it for?
http://aix360.mybluemix.net/consumer
IBM Research Case Study: AI-powered credit approval system
Audit and report if the model is treating
different subgroups “fairly”
Check stability of the models predictions
Cross-check if the features make sense
Introduce improvements to the model
Actively monitor model performance
...

Data scientist’s view on XAI
https://www.kaggle.com/dansbecker/advanced-uses-of-shap-values https://fredhohman.com/papers/chameleon

Everybody else
https://www.kaggle.com/dansbecker/advanced-uses-of-shap-values https://fredhohman.com/papers/chameleon

What is XAI?
It looks like what XAI is has a lot to do with humans!
(It’s a mess…)
(...but a mess we know how to deal with!)

Let’s go back a step

What is AI?
https://developer.nvidia.com/discover/convolutional-neural-network

What is machine learning?
data
rules
resultscomputation
Traditional programming
data
rules
results
Machine learning
computation
Training the model = process of
discovering the rules
Data, results = sets of numbers
Rules = mathematical transformations

Let’s do it together!
% nuts
price
Let’s build a cookie classifier!
https://medium.com/@kozyrkov/machine-learning-is-the-emperor-wearing-clothes-928fe406fe09
Example borrowed from Cassie Kozyrkov

Let’s do it together!
Maybe it’s a cookie with goji berries?
Or maybe I wrote down 9 instead of 3?
% nuts
price
Let’s build a cookie classifier!

Find a line separating the two classes
Decision
trees
SVMs Neural
nets

Pick the best line
Train the model

Pick the best line

Count the mistakes
Y N
Y 11 2
N 1 17
Predicted
Ground Truth
Confusion Matrix: false positives,
false negatives, sensitivity,
precision, specificity, accuracy…
Test Set vs Train Set: Hold out
some “fresh” data for this!

Save the model

Use the model on new data
?
? N
Prediction:
Confidence: how sure is the
model about this prediction
What could explainability mean here?

Translating between the two worlds
Human
reality
Mathematical
model
Will I like
them?
ML
Classifier
Cookies
Set of
numbers
Model
prediction
Probably
not

Translating between the two worlds
Human
reality
Mathematical
model
Use case,
problem
ML task
Reality
Data Numerical
result
Insight

For XAI, I need to translate again
Human
reality
Mathematical
model
Use case,
problem
ML task
Reality
Data Numerical
result
Insight Why
Algorithm Numerical
result
Explanation

The most important question is about people
Human
reality
Mathematical
model
Use case,
problem
ML task
Reality
Data Numerical
result
Insight Why
Algorithm Numerical
result
Explanation
Did it increase reliability, trust, safety?
Did it influence what a user did?
If we can design AI which is “reliable, trusted,
and safe” we can dramatically enhance human
performance in the coming decade.

- Data is never a perfect representation of reality
- There is never a clear boundary - there are always points that are
misidentified
- Different algorithms will make their “decision” using different
criteria
- There may be more to “explain” than the algorithm itself (data
collection, training process etc.)
- If there is a signal, AI will pick it up
What does it mean for us?

AI is really stupid
https://www.topbots.com/chihuahua-muffin-searching-best-computer-vision-api/
https://arxiv.org/abs/1602.04938
https://deepai.org/publication/understanding-unequal-gender-classification-accuracy-from-face-images
It only sees a surface It can be very confident for a
wrong reason
If there is a signal, it will find it
Wolf → White background detector Gender → Make-up detector

What can we do?

What can we do?
Interrogate the model
What did you notice?
What did you think?
Work with the context
and borrow best practices

“What did you notice?” Feature importances, activation maps,
saliency maps, attributions, feature maps
Cookie classifier:
- % nuts
- price
- % raisins
- producer
- size
- density
- sweetness
- # chocolate chips
- % seeds
- ...
Explanation:
- % nuts: 60
- price: 10
- % raisins: 13
- producer: 2
- size: 7
- density: 22
- sweetness: 10
- # chocolate chips: 90
- % seeds: 21
- ...
Prediction:
I would like it
Input new cookie:
3% nuts, 1.5 CHF,
20% raisins...

“What did you notice?” Feature importances, activation maps,
saliency maps, attributions, feature maps
Explanation:
- % nuts: 60
- price: 10
- % raisins: 13
- producer: 2
- size: 7
- density: 22
- sweetness: 10
- % seeds: 21
- ...
LIME: locally
interpretable
explanations
Shapley
values / SHAP:
game theory
Attention, anchors, distillation,
randomized input sampling...
Many clever methods to get these numbers

It also works for images...
https://arxiv.org/abs/1806.07421
http://shared-interest.csail.mit.edu
Shared Interest - Saliency
Explorer (MIT)
Such insights are actionable!

...and for text
https://arxiv.org/abs/2009.05818 Classifying reviews,
detecting emotions,
automatizing client
complaints, triaging
documents...

Information visualization matters
Explanation:
- % nuts: 60
- price: 10
- % raisins: 13
- producer: 2
- size: 7
- density: 22
- sweetness: 10
- % seeds: 21
- ...

Explanation:
- % nuts: 60
- price: 10
- % raisins: 13
- producer: 2
- size: 7
- density: 22
- sweetness: 10
- % seeds: 21
- ...
Maybe I’m a cookie domain
expert and expect
sweetness to be important

Explanation:
- % nuts: 60
- price: 10
- % raisins: 13
- producer: 2
- size: 7
- density: 22
- sweetness: 10
- % seeds: 21
- ...
Maybe I’m a cookie domain
expert and expect
sweetness to be important
Maybe I am visually impaired and need completely another
form of accessible data visualization?
Maybe in my daily work I always use a particular chart type?

Context: is 3.6 good or bad?
Measurement
Range, Unit -
Context
Accuracy -
Uncertainty
Danger -
Context

https://www.youtube.com/watch?v=pdUTdaZpses
David McCandless /
Information is Beautiful
Context and baselines
Baseline could be statistical,
or it could be another example
that you can compare to
(error bars!)

Leverage mental models
https://www.youtube.com/watch?v=a7dhEpx6tfE
Jack the Checker
(Evelyn Muenster /
Chart Doktor)

Many AI use cases are like medical tests
Am I pregnant?
Am I eligible for a mortgage?
Is this transaction fraudulent?
Will my customer terminate the subscription?
Is this person a likely reoffender?
...
How does my
pregnancy test work?
Am I pregnant,
if the test result was X?

https://www.bmj.com/content/369/bmj.m1808

https://www.bmj.com/content/369/bmj.m1808
...And you could think that since medical tests
are so common, we can explain them well!

Some strategies
- Interrogate the model
- Leverage context
- Leverage information visualization
- Leverage existing approaches
- Leverage mental models
- ...

Take home message

(X)AI is a mess AI is stupid
But it is still very useful!
Make friends with your
fellow data scientists
And...

Thank you!
info@teresa-kubacka.com

Explainable AI for Ordinary Humans - Teresa Kubacka

Recommended

Recommended

More Related Content

Similar to Explainable AI for Ordinary Humans - Teresa Kubacka

Similar to Explainable AI for Ordinary Humans - Teresa Kubacka (20)

Recently uploaded

Recently uploaded (20)

Explainable AI for Ordinary Humans - Teresa Kubacka