Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Why do we need
XAI?
https://www.wired.com/2015/10/can-learn-epic-failure-google-flu-trends/
https://www.massdevice.com/report-ibm-watson-delivered-unsafe-and-inaccurate-
cancer-recommendations/
https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm
!10
Meet DALEX
How to explain visually?
1/ Why bother?
Data visualization is a visual representation
of abstract data to amplify cognition
stuart card
Data visualization is a visual representation
of abstract data to amplify cognition
stuart card
2/ The goals
1/ readability
2/ accessibility
3/ consistency
1/ readability
2/ accessibility
3/ consistency
efficiency
3/ The changes
svm
rf
lm
250 500 750 1000 1250
_full_model_
no.rooms
construction.year
floor
surface
district
_baseline_
_full_model_
no....
3000
3200
3400
surface
prediction
3600
3800
Surface
6019 100 148
Random Forest
Random Forest
Factor Merger
Srodmiescie
Och...
feature influence
Variables attributions
GBM
intercept
district: Srodmiescie
surface: 22
no.rooms: 2
construction.year: 200...
FactorMerger
FactorMerger
FactorMerger
Srodmiescie
Ochota
Mokotow
Zoliborz
Ursus
Bielany
Bemowo
Wola
Ursynow
Praga
0 2000 4000
GROUP FREQUENCYNAME P...
Ceteris Paribus Profiles
Ceteris Paribus Profiles
3000
2900
3200
3400
surface
prediction
3600
3800
District impact on surface
6020 100 144
Bemowo Mo...
Ceteris Paribus Profiles
3000
2900
3200
3400
surface
prediction
3600
3800
District impact on surface
6020 100 144
Bemowo Mo...
The colors
#8bdcbe
Main colors
#f05a71
#371ea3
#46bac2
#ae2c87
#ffa58c
#4378bf
#160e3b
Additional colors
#f0f0f4
#ceced9
m...
The colors
#8bdcbe
Main colors
#f05a71
#371ea3
#46bac2
#ae2c87
#ffa58c
#4378bf
#160e3b
Additional colors
#f0f0f4
#ceced9
m...
The colors
#8bdcbe
Main colors
#f05a71
#371ea3
#46bac2
#ae2c87
#ffa58c
#4378bf
#160e3b
Additional colors
#f0f0f4
#ceced9
m...
The colors
https://www.economist.com/finance-and-economics/2016/09/03/more-spend-less-thrift
Deuteranopia simulationOrigina...
The colors
#8bdcbe
#8bdcbe
position: 25%
#c7f5bf
position: 0%
#f05a71 #371ea3 #46bac2
#46bac2
position: 50%
#ae2c87 #ffa58...
The colors
label
label
label
x axis title
yaxistitle
label
label
Chart Title
labellabel label label label
25
30
8
15
20
25
30
8
15
20...
The proportions
8 8
1000489 2000 3000 4043
life_lenght
residuals
Residuals vs life lenght
0
250
500
750
otherRandom Forest...
The proportions
minimal margin
fixed margin
Axis Labels:
Fira Sans Regular 11pt
Axis Title:
Fira Sans Regular 13pt
8
10
15
...
Variable importance
GBM
baseline
ditrict
surface
floor
construction.year
no. rooms
full model
Drop-out loss
250 500 750 100...
30
8
10
Variable importance
GBM
baseline
ditrict
surface
floor
construction.year
no. rooms
full model
Drop-out loss
250 500...
minimal margin
fixed margin
25
30
8
10
8
10
15
20
Variable importance
GBM
baseline
ditrict
surface
floor
construction.year
n...
1/ final effect = needs + goals + constraints*
* plus a bit of fun and looking at other beautiful charts for inspiration
1/ final effect = needs + goals + constraints*
2/ solutions <- “why” & “what for” questions
* plus a bit of fun and looking...
1/ final effect = needs + goals + constraints*
2/ solutions <- “why” & “what for” questions
3/ science + design = ∞
* plus ...
Meet DALEX 2.0
Predictive Models: Visual Exploration, Explanation and Debugging
Production
Development
Concept ValidateForge
Model debugg...
https://www.encyclopedia-titanica.org/
What are the odds of surviving?
What next?
https://kmichael08.github.io
https://github.com/MI2DataLab/modelDown
https://chudekm.shinyapps.io/model_explorer_example/
Thank you
Machine learning meets Design. Design meets Machine learning.
Machine learning meets Design. Design meets Machine learning.
Machine learning meets Design. Design meets Machine learning.
Machine learning meets Design. Design meets Machine learning.
Machine learning meets Design. Design meets Machine learning.
Machine learning meets Design. Design meets Machine learning.
Machine learning meets Design. Design meets Machine learning.
Machine learning meets Design. Design meets Machine learning.
Machine learning meets Design. Design meets Machine learning.
Machine learning meets Design. Design meets Machine learning.
Machine learning meets Design. Design meets Machine learning.
Machine learning meets Design. Design meets Machine learning.
Machine learning meets Design. Design meets Machine learning.
Machine learning meets Design. Design meets Machine learning.
Machine learning meets Design. Design meets Machine learning.
Machine learning meets Design. Design meets Machine learning.
Machine learning meets Design. Design meets Machine learning.
Machine learning meets Design. Design meets Machine learning.
Machine learning meets Design. Design meets Machine learning.
Upcoming SlideShare
Loading in …5
×

Machine learning meets Design. Design meets Machine learning.

620 views

Published on

eXplainable Artificial Intelligence is supposed to be easy to understand for humans. This means that it needs to be properly design. In this presentation Przemek shows why XAI is an important topic and Hanna shows how to design visual presentation for predictive models. Presented by Przemyslaw Biecek and Hanna Piotrowska at satRdays Gdansk 2019

Published in: Data & Analytics
  • Be the first to comment

Machine learning meets Design. Design meets Machine learning.

  1. 1. Why do we need XAI?
  2. 2. https://www.wired.com/2015/10/can-learn-epic-failure-google-flu-trends/
  3. 3. https://www.massdevice.com/report-ibm-watson-delivered-unsafe-and-inaccurate- cancer-recommendations/
  4. 4. https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm
  5. 5. !10
  6. 6. Meet DALEX
  7. 7. How to explain visually?
  8. 8. 1/ Why bother?
  9. 9. Data visualization is a visual representation of abstract data to amplify cognition stuart card
  10. 10. Data visualization is a visual representation of abstract data to amplify cognition stuart card
  11. 11. 2/ The goals
  12. 12. 1/ readability 2/ accessibility 3/ consistency
  13. 13. 1/ readability 2/ accessibility 3/ consistency efficiency
  14. 14. 3/ The changes
  15. 15. svm rf lm 250 500 750 1000 1250 _full_model_ no.rooms construction.year floor surface district _baseline_ _full_model_ no.rooms construction.year floor surface district _baseline_ _full_model_ no.rooms construction.year floor surface district _baseline_ Drop−out loss ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ●●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ●● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ●● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ●●●● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ● ●● ●●● ● ● ● ●● ● ●●● ● ● ● ●●● ●● ● ● ● ● ● ● ●● ● ●● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ● ●● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ●● ● ●●●● ● ●● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ●● ●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●●●●●●● ● ● ●●● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ●●●● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ●●● ●● ●● ● ● ● ● ● ●● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ●● ● ●●●●● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●●● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ●●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ●● ●●● ● ●● ● ● ● ●●●● ● ●●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ●● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●●● ●●● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ●● ●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ●● ●● ● ● ● ● ● ●● ● ● ● ●● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0 250 500 750 1000 2000 3000 4000 life_length residuals label ● ● lm rf Residuals vs life_lengthsurface 40 80 120 3000 3300 3600 ʻ_x_ʻ ʻ_yhat_ʻ hours 40 50 60 70 80 0.00 0.25 0.50 0.75 ʻ_x_ʻ ʻ_yhat_ʻ ʻ_label_ʻ fired ok promoted
  16. 16. 3000 3200 3400 surface prediction 3600 3800 Surface 6019 100 148 Random Forest Random Forest Factor Merger Srodmiescie Ochota Mokotow Zoliborz Ursus Bielany Bemowo Wola Ursynow Praga 0 2000 4000 GROUP FREQUENCYNAME PRICE MEAN 5109.19 3954.83 3946.96 3918.55 3058.52 3045.79 3028.58 3011.69 3009.72 2991.48 NR 1 2 2 3 4 4 5 6 6 7feature influence Variables attributions GBM intercept district: Srodmiescie surface: 22 no.rooms: 2 construction.year: 2005 floor: 1 prediction 2000 2100 2200 2300 2400 2500 2600 2700 2046 2614.9 +358 +160 +78 -39.5 +12.4 Drop-out loss Variable importance GBM baseline ditrict surface floor construction.year no. rooms full model 250 500 750 1000 1250 Distribution of residuals residualspercentage 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 100% 5000 1000 1500 2000 2500 rf gm m3 Distribution of residuals lm mod1 lr Model Ranking Model 01 Model 02 Model 03 Model 04 Model 05 Model 06 invSC invSC invREC invROCC invMAE invMSE invREC invROCC invMAE invMSE invMAE 0.81 0.68 1 0.99 0.7 0.47 0.59 1 0.98 1 0.71 1 0.98 1 1 0.5 1 0.82 0.81 0.65 0.990.72 0.63 1 0.7 0.7 0.91 0.7 0.7 0.7 0.85 0.780.82 0.8 0.5 0.5 0.62 0.69 1 0.65 0.69 0.65 10.82 0.63 0.8 1 0.8 0.87 0.7 1 0.7 1 0.81 0.63 1 0.7 0.8 0.6 1 1 0.69 0.98 1000489 2000 3000 4043 life_lenght residuals Residuals vs life lenght 0 250 500 750 GMB Random Forest LM 40 80 100 60 1920 1940 1960 1980 2000 construction.year surface 120 Surface vs construction year 5000 4000 2000 3000 1000 Prediction & residual value residual
  17. 17. feature influence Variables attributions GBM intercept district: Srodmiescie surface: 22 no.rooms: 2 construction.year: 2005 floor: 1 prediction 2000 2100 2200 2300 2400 2500 2600 2700 2046 2614.9 +358 +160 +78 -39.5 +12.4 BreakDown
  18. 18. FactorMerger
  19. 19. FactorMerger
  20. 20. FactorMerger Srodmiescie Ochota Mokotow Zoliborz Ursus Bielany Bemowo Wola Ursynow Praga 0 2000 4000 GROUP FREQUENCYNAME PRICE MEAN 5109.19 3954.83 3946.96 3918.55 3058.52 3045.79 3028.58 3011.69 3009.72 2991.48 NR 1 2 3 3 4 4 4 4 4 4 Random Forest LM Factor Merger Srodmiescie Ochota Mokotow Zoliborz Ursus Bielany Bemowo Wola Ursynow Praga 0 2000 4000 GROUP FREQUENCYNAME PRICE MEAN 5109.19 3954.83 3946.96 3918.55 3058.52 3045.79 3028.58 3011.69 3009.72 2991.48 NR 1 2 2 3 4 4 5 6 6 7
  21. 21. Ceteris Paribus Profiles
  22. 22. Ceteris Paribus Profiles 3000 2900 3200 3400 surface prediction 3600 3800 District impact on surface 6020 100 144 Bemowo Mokotow Ochota Srodmiescie Ursus
  23. 23. Ceteris Paribus Profiles 3000 2900 3200 3400 surface prediction 3600 3800 District impact on surface 6020 100 144 Bemowo Mokotow Ochota Srodmiescie Ursus
  24. 24. The colors #8bdcbe Main colors #f05a71 #371ea3 #46bac2 #ae2c87 #ffa58c #4378bf #160e3b Additional colors #f0f0f4 #ceced9 model2lrmod1lm rf gm m3
  25. 25. The colors #8bdcbe Main colors #f05a71 #371ea3 #46bac2 #ae2c87 #ffa58c #4378bf #160e3b Additional colors #f0f0f4 #ceced9 model2lrmod1lm rf gm m3
  26. 26. The colors #8bdcbe Main colors #f05a71 #371ea3 #46bac2 #ae2c87 #ffa58c #4378bf #160e3b Additional colors #f0f0f4 #ceced9 model2lrmod1lm rf gm m3
  27. 27. The colors https://www.economist.com/finance-and-economics/2016/09/03/more-spend-less-thrift Deuteranopia simulationOriginal graphic
  28. 28. The colors #8bdcbe #8bdcbe position: 25% #c7f5bf position: 0% #f05a71 #371ea3 #46bac2 #46bac2 position: 50% #ae2c87 #ffa58c #4378bf #4378bf position: 75% Main colors Gradients #160e3b #f0f0f4#ceced9 alpha 50% Additional colors #c7f5bf #b3eebe #9fe5bd #8bdcbe #77d1be #61c5c0 #46bac2 #45a6c4 #4590c4 #4378bf #415fb9 #3d42af #371ea3 #371ea3 position: 100% #77d1be position: 25% #9fe5bd position: 0% #46bac2 position: 50% #4590c4 position: 75% #9fe5bd #92debd #84d8be #77d1be #67c9bf #56c2c1 #46bac2 #46acc3 #459ec3 #4590c4 #4480c0 #426fbd #371ea3 #371ea3 position: 100%
  29. 29. The colors
  30. 30. label label label x axis title yaxistitle label label Chart Title labellabel label label label 25 30 8 15 20 25 30 8 15 20 81520 20 The proportions
  31. 31. The proportions 8 8 1000489 2000 3000 4043 life_lenght residuals Residuals vs life lenght 0 250 500 750 otherRandom Forest Chart panel: aspect ratio 3:2 Chart panel center aligned horizontally 25 30 8 minimal margin fixed margin 8 15 20 81520 20 Title: Fira Sans SemiBold 18pt Legend: Fira Sans Regular 11pt Axis Title: Fira Sans Regular 13pt Axis Title: Fira Sans Regular 13pt Axis Labels: Fira Sans Regular 11pt Axis Labels: Fira Sans Regular 11pt Line: 2px #371ea3 Dots: 2px #46bac2Inactive: #ceced9 50%
  32. 32. The proportions minimal margin fixed margin Axis Labels: Fira Sans Regular 11pt Axis Title: Fira Sans Regular 13pt 8 10 15 20 820 20 Title: Fira Sans SemiBold 18pt Small multiple title: Fira Sans SemiBold 13pt Axis Labels: Fira Sans Regular 11pt Prediction Label: Fira Sans SemiBold 11pt 25 30 8 10 Bars: 12 px Space: 6 px feature influence Variables attributions GBM intercept district: Srodmiescie surface: 22 no.rooms: 2 construction.year: 2005 floor: 1 prediction 2000 2100 2200 2300 2400 2500 2600 2700 2046 2614.9 +358 +160 +78 -39.5 +12.4 Values: Fira Sans Regular 11pt white background Prediction value: Fira Sans SemiBold 11pt, white background Intercept line: 0.5px, dash: 1px, space: 2px Line: 0.5px
  33. 33. Variable importance GBM baseline ditrict surface floor construction.year no. rooms full model Drop-out loss 250 500 750 1000 1250 Bars: 12 px Space: 6 px Bars: 12 px Space: 6 px feature influence Variables attributions GBM intercept district: Srodmiescie surface: 22 no.rooms: 2 construction.year: 2005 floor: 1 prediction 2000 2100 2200 2300 2400 2500 2600 2700 2046 2614.9 +358 +160 +78 -39.5 +12.4 Random Forest Factor Merger Srodmiescie Ochota Mokotow Zoliborz Ursus Bielany Bemowo Wola Ursynow Praga 0 2000 4000 GROUP FREQUENCYNAME PRICE MEAN 5109.19 3954.83 3946.96 3918.55 3058.52 3045.79 3028.58 3011.69 3009.72 2991.48 NR 1 2 2 3 4 4 5 6 6 7 Bars: 12 px Space: 6 px
  34. 34. 30 8 10 Variable importance GBM baseline ditrict surface floor construction.year no. rooms full model Drop-out loss 250 500 750 1000 1250 Bars: 12 px Space: 6 px 30 8 10 Bars: 12 px Space: 6 px feature influence Variables attributions GBM intercept district: Srodmiescie surface: 22 no.rooms: 2 construction.year: 2005 floor: 1 prediction 2000 2100 2200 2300 2400 2500 2600 2700 2046 2614.9 +358 +160 +78 -39.5 +12.4 8 10 Random Forest Factor Merger Srodmiescie Ochota Mokotow Zoliborz Ursus Bielany Bemowo Wola Ursynow Praga 0 2000 4000 GROUP FREQUENCYNAME PRICE MEAN 5109.19 3954.83 3946.96 3918.55 3058.52 3045.79 3028.58 3011.69 3009.72 2991.48 NR 1 2 2 3 4 4 5 6 6 7 30 8 10 8 10 Bars: 12 px Space: 6 px
  35. 35. minimal margin fixed margin 25 30 8 10 8 10 15 20 Variable importance GBM baseline ditrict surface floor construction.year no. rooms full model Drop-out loss 250 500 750 1000 1250 Bars: 12 px Space: 6 px 20 25 30 8 10 Bars: 12 px Space: 6 px feature influence Variables attributions GBM intercept district: Srodmiescie surface: 22 no.rooms: 2 construction.year: 2005 floor: 1 prediction 2000 2100 2200 2300 2400 2500 2600 2700 2046 2614.9 +358 +160 +78 -39.5 +12.4 8 10 Random Forest Factor Merger Srodmiescie Ochota Mokotow Zoliborz Ursus Bielany Bemowo Wola Ursynow Praga 0 2000 4000 GROUP FREQUENCYNAME PRICE MEAN 5109.19 3954.83 3946.96 3918.55 3058.52 3045.79 3028.58 3011.69 3009.72 2991.48 NR 1 2 2 3 4 4 5 6 6 7 25 30 8 10 8 10 Bars: 12 px Space: 6 px 20
  36. 36. 1/ final effect = needs + goals + constraints* * plus a bit of fun and looking at other beautiful charts for inspiration
  37. 37. 1/ final effect = needs + goals + constraints* 2/ solutions <- “why” & “what for” questions * plus a bit of fun and looking at other beautiful charts for inspiration
  38. 38. 1/ final effect = needs + goals + constraints* 2/ solutions <- “why” & “what for” questions 3/ science + design = ∞ * plus a bit of fun and looking at other beautiful charts for inspiration
  39. 39. Meet DALEX 2.0
  40. 40. Predictive Models: Visual Exploration, Explanation and Debugging Production Development Concept ValidateForge Model debugging Model development is an iterative process. Each iteration brings new insights. Early phases: Crisp modeling, general understanding of the problem. Medium phases: Selective modeling, here we select the best type of model. Late phases: Fine tuning of model parameters or variable engineering . In each iteration model development starts with some concepts, ideas, then the model is trained and finally model needs to be validated. Predictions need to be explained. Here the instance level explanation helps. With time the model performance may deteriorate, thus it requires constant monitoring, e.g. with the drifter package. Drop-out loss Variable importance GBM baseline ditrict surface floor construction.year no. rooms full model 250 500 750 1000 1250 3000 3200 3400 surface prediction 3600 3800 Surface 6019 100 148 Random Forest feature influence Variables attributions GBM intercept district: Srodmiescie surface: 22 no.rooms: 2 construction.year: 2005 floor: 1 prediction Random Forest intercept district: Srodmiescie surface: 22 no.rooms: 2 construction.year: 2005 floor: 1 prediction LM intercept district: Srodmiescie surface: 22 no.rooms: 2 construction.year: 2005 floor: 1 prediction 2000 2100 2200 2300 2400 2500 2600 2700 2800 2900 2046 2614.9 +358 +160 +78 -39.5 +12.4 2800 2425 -338 -112 +74 -53 +26 2378 2324.9 -239 +160 +68 +39.5 -12.4 feature influence Variables attributions GBM intercept district: Srodmiescie surface: 22 no.rooms: 2 construction.year: 2005 floor: 1 prediction 2000 2100 2200 2300 2400 2500 2600 2700 2046 2614.9 +358 +160 +78 -39.5 +12.4 3000 3200 3400 surface prediction 3600 3800 Surface 6019 100 148 Random Forest 3000 3200 3400 2 4 6 prediction 3600 3800 3000 3200 3400 3600 3800 Surface 1920 1940 1960 1980 2010 no.roomsfloor surfaceconstruction.year 4020 80 120 144 2.50.8 1.15.0 7.5 11.2 Variable selection Feature engineering Random Forest Factor Merger Srodmiescie Ochota Mokotow Zoliborz Ursus Bielany Bemowo Wola Ursynow Praga 0 2000 4000 group frequencyname price mean 5109.19 3954.83 3946.96 3918.55 3058.52 3045.79 3028.58 3011.69 3009.72 2991.48 nr 1 2 2 3 4 4 5 6 6 7 Prediction explanations What-If analysis Concept drift detection explain
  41. 41. https://www.encyclopedia-titanica.org/ What are the odds of surviving?
  42. 42. What next?
  43. 43. https://kmichael08.github.io
  44. 44. https://github.com/MI2DataLab/modelDown
  45. 45. https://chudekm.shinyapps.io/model_explorer_example/
  46. 46. Thank you

×