Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Age(years)
Explaining	Machine	Learning
Model	Predictions
Scott	Lundberg
Why	do	we	care	so	much	about	explainability
in	machine	learning?
model 55%
chance	John	will	have	
repayment	problems
John,	a	bank	customer
No	loan
Why?!
Why?! AI	magic!
Interpretable Accurate
Complex	model ✘ ✔
Simple model ✔ ✘
Interpretable	or	accurate:	choose	one.	
😀	⚖ 💰
3
Complex	models	are	
inherently	complex!
But	a	single	prediction	involves	only	a	
small	piece	of	that	complexity.
Input	val...
6
How	did	we	get	here?
Base	rate Prediction	for	John
20% 55%
7
Base	rate
Work	experience	=	1	yr
20% 35%
Day	trader Open	accounts	=	1
70%
Capital	gains
90%55%
8
The	order	matters!
Work	experience	=	1	yr
Day	trader
Nobel	Prize	in	2012
Lloyd	Shapley
9
SHapley Additive	exPlanation (SHAP)	values
Age	=	20
Day	trader
Shapley	values	result	from	averaging	over	all	N!	possible...
10
Mortality	risk	model
Global feature importance Local explanation summary(A)
(log relative risk of mortality)
Mortality model
(F/M)
Mortality	ri...
Global feature importance Local explanation summary(A)
(log relative risk of mortality)
Mortality model
(F/M)
Mortality	ri...
Global feature importance Local explanation summary(A)
(log relative risk of mortality)
Mortality model
(F/M)
Mortality	ri...
Global feature importance Local explanation summary(A)
(log relative risk of mortality)
Mortality model
(F/M)
Mortality	ri...
Global feature importance Local explanation summary(A)
(log relative risk of mortality)
Mortality model
(F/M)
Mortality	ri...
Global feature importance Local explanation summary(A)
(log relative risk of mortality)
Mortality model
(F/M)
Mortality	ri...
Reveal	rare	high-magnitude	mortality	effects
Global feature importance Local explanation summary(A)
(log relative risk of ...
Reveal	rare	high-magnitude	mortality	effects
Global feature importance Local explanation summary(A)
(log relative risk of ...
Reveal	rare	high-magnitude	mortality	effects
Global feature importance Local explanation summary(A)
(log relative risk of ...
Reveal	rare	high-magnitude	mortality	effects
Global feature importance Local explanation summary(A)
(log relative risk of ...
Dependence	plots	reveal	the	increased	
danger	of	early	onset	high	blood	pressure (C)
SHAPvalueforsystolicbloodpressure
wit...
The	varying	risk	of	sex	over	a	lifetime
22
23
Model	Monitoring
Model	monitoring
Time
Training	performance Test	performance
Can	you	find	where	we	introduced	the	bug?
24
Model	monitoring
Now	can	you	find	where	we	introduced	the	bug?
25
False True
Model	monitoring
Time
Transient	electronic	medical	record
Time
26
False True
Model	monitoring
Time
Gradual	change	in	atrial	fibrillation
ablation	procedure	durations
Time
27
False True
Don’t	take	my	word	for	it,	try	it	yourself	J
github.com/slundberg/shap
github.com/slundberg/shap
…
Don’t	take	my	word	for	it,	try	it	yourself	J
30
Important	questions	to	ask	when	using	SHAP:
1. If	you	are	using	a	model	agnostic	explainer,	have	you	drawn	
enough	samp...
Thanks!
You’ve finished this document.
Download and read it offline.
Upcoming SlideShare
What to Upload to SlideShare
Next
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

Share

Rsqrd AI: Exploring Machine Learning Model Predictions

Download to read offline

In this talk, Scott Lundberg from Microsoft Research talks about explainability in ML and SHAP.

Presented 09/25/2019

**These slides are from a talk given at Rsqrd AI. Learn more at rsqrdai.org**

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

Rsqrd AI: Exploring Machine Learning Model Predictions

  1. 1. Age(years) Explaining Machine Learning Model Predictions Scott Lundberg
  2. 2. Why do we care so much about explainability in machine learning?
  3. 3. model 55% chance John will have repayment problems John, a bank customer No loan Why?! Why?! AI magic!
  4. 4. Interpretable Accurate Complex model ✘ ✔ Simple model ✔ ✘ Interpretable or accurate: choose one. 😀 ⚖ 💰 3
  5. 5. Complex models are inherently complex! But a single prediction involves only a small piece of that complexity. Input value Output value 5
  6. 6. 6 How did we get here? Base rate Prediction for John 20% 55%
  7. 7. 7 Base rate Work experience = 1 yr 20% 35% Day trader Open accounts = 1 70% Capital gains 90%55%
  8. 8. 8 The order matters! Work experience = 1 yr Day trader Nobel Prize in 2012 Lloyd Shapley
  9. 9. 9 SHapley Additive exPlanation (SHAP) values Age = 20 Day trader Shapley values result from averaging over all N! possible orderings. (NP-hard)
  10. 10. 10 Mortality risk model
  11. 11. Global feature importance Local explanation summary(A) (log relative risk of mortality) Mortality model (F/M) Mortality risk model
  12. 12. Global feature importance Local explanation summary(A) (log relative risk of mortality) Mortality model (F/M) Mortality risk model
  13. 13. Global feature importance Local explanation summary(A) (log relative risk of mortality) Mortality model (F/M) Mortality risk model
  14. 14. Global feature importance Local explanation summary(A) (log relative risk of mortality) Mortality model (F/M) Mortality risk model
  15. 15. Global feature importance Local explanation summary(A) (log relative risk of mortality) Mortality model (F/M) Mortality risk model
  16. 16. Global feature importance Local explanation summary(A) (log relative risk of mortality) Mortality model (F/M) Mortality risk model
  17. 17. Reveal rare high-magnitude mortality effects Global feature importance Local explanation summary(A) (log relative risk of mortality) Mortality model (F/M) Conflates the prevalence of an effect with the magnitude of an effect Mortality risk model
  18. 18. Reveal rare high-magnitude mortality effects Global feature importance Local explanation summary(A) (log relative risk of mortality) Mortality model (F/M)
  19. 19. Reveal rare high-magnitude mortality effects Global feature importance Local explanation summary(A) (log relative risk of mortality) Mortality model (F/M) Rare high magnitude effects
  20. 20. Reveal rare high-magnitude mortality effects Global feature importance Local explanation summary(A) (log relative risk of mortality) Mortality model (F/M) Lots of ways to die young…Not many ways to live longer…
  21. 21. Dependence plots reveal the increased danger of early onset high blood pressure (C) SHAPvalueforsystolicbloodpressure withouttheageinteraction (logrelativeriskofmortality) Sys (B) (logrelativeriskofmortality) Systolic blood pressure (mmHg) Age(years) Kidney model = Vertical dispersion is driven by interaction effects 21
  22. 22. The varying risk of sex over a lifetime 22
  23. 23. 23 Model Monitoring
  24. 24. Model monitoring Time Training performance Test performance Can you find where we introduced the bug? 24
  25. 25. Model monitoring Now can you find where we introduced the bug? 25 False True
  26. 26. Model monitoring Time Transient electronic medical record Time 26 False True
  27. 27. Model monitoring Time Gradual change in atrial fibrillation ablation procedure durations Time 27 False True
  28. 28. Don’t take my word for it, try it yourself J github.com/slundberg/shap
  29. 29. github.com/slundberg/shap … Don’t take my word for it, try it yourself J
  30. 30. 30 Important questions to ask when using SHAP: 1. If you are using a model agnostic explainer, have you drawn enough samples? 2. What background population are you using to estimate the effect of a feature being “missing”? 3. What model output are you explaining? (log-odds, probability, rank-order, etc.) 4. Are you perturbing a model in ways that don’t make sense? (dealing with tightly correlated features)
  31. 31. Thanks!

In this talk, Scott Lundberg from Microsoft Research talks about explainability in ML and SHAP. Presented 09/25/2019 **These slides are from a talk given at Rsqrd AI. Learn more at rsqrdai.org**

Views

Total views

57

On Slideshare

0

From embeds

0

Number of embeds

10

Actions

Downloads

0

Shares

0

Comments

0

Likes

0

×