Downloaded 159 times
![6
How did we get here?
Base rate Prediction for John
22%
𝑓 𝑥0
16%
𝐸[𝑓 𝑋 ]](https://image.slidesharecdn.com/1140amgpscottlundbergfinalv2-191022184911/75/Scott-Lundberg-Microsoft-Research-Explainable-Machine-Learning-with-Shapley-Values-H2O-World-NYC-2019-6-2048.jpg)
![7
16% 22%
𝐸[𝑓 𝑋 ] 𝑓 𝑥0
Base rate
𝜙0
Income not verified
18.2%
𝐸 𝑓 𝑋 𝑑𝑜(𝑋1 = 𝑥1)]
𝜙1
No recent account openings
21%
𝐸 𝑓 𝑋 𝑑𝑜(𝑋1,2 = 𝑥1,2)]
22.5%
𝐸 𝑓 𝑋 𝑑𝑜(𝑋1,2,3 = 𝑥1,2,3)]
18.5%
𝐸 𝑓 𝑋 𝑑𝑜(𝑋1,2,3,4 = 𝑥1,2,3,4)]
𝜙2
𝜙3𝜙4
𝜙5
DTI = 30
Delinquent 10 months ago
46 years of credit history
The order matters!
*Janzing et al. 2019
*](https://image.slidesharecdn.com/1140amgpscottlundbergfinalv2-191022184911/75/Scott-Lundberg-Microsoft-Research-Explainable-Machine-Learning-with-Shapley-Values-H2O-World-NYC-2019-7-2048.jpg)
![8
𝐸[𝑓 𝑋 ] 𝑓 𝑥0
𝜙0
𝜙1
No recent account openings
𝜙2
𝜙3
𝜙4
𝜙5
46 years of credit history
The order matters!
Nobel Prize in 2012
Lloyd Shapley](https://image.slidesharecdn.com/1140amgpscottlundbergfinalv2-191022184911/75/Scott-Lundberg-Microsoft-Research-Explainable-Machine-Learning-with-Shapley-Values-H2O-World-NYC-2019-8-2048.jpg)
![9
𝐸[𝑓 𝑋 ] 𝑓 𝑥0
𝜙0
𝜙1
𝜙2
𝜙3
𝜙4
𝜙5
Shapley properties
Additivity (local accuracy) – The sum of the local
feature attributions equals the difference between
rate and the model output.
1](https://image.slidesharecdn.com/1140amgpscottlundbergfinalv2-191022184911/75/Scott-Lundberg-Microsoft-Research-Explainable-Machine-Learning-with-Shapley-Values-H2O-World-NYC-2019-9-2048.jpg)
![10
𝐸[𝑓 𝑋 ] 𝑓 𝑥0
𝜙0
𝜙1
𝜙2
𝜙3
𝜙4
𝜙5
Shapley properties
Monotonicity (consistency) – If you change the
original model such that a feature has a larger
possible ordering, then that input’s attribution
decrease.
2
Violating consistency means you can’t trust feature orderings
based on your attributions.
…even within the same model!](https://image.slidesharecdn.com/1140amgpscottlundbergfinalv2-191022184911/75/Scott-Lundberg-Microsoft-Research-Explainable-Machine-Learning-with-Shapley-Values-H2O-World-NYC-2019-10-2048.jpg)
![11
𝐸[𝑓 𝑋 ] 𝑓 𝑥0
𝜙0
𝜙1
𝜙2
𝜙3
𝜙4
𝜙5
Shapley values result from averaging over all N! possible orderings.
(NP-hard)](https://image.slidesharecdn.com/1140amgpscottlundbergfinalv2-191022184911/75/Scott-Lundberg-Microsoft-Research-Explainable-Machine-Learning-with-Shapley-Values-H2O-World-NYC-2019-11-2048.jpg)
![ex = shap.TreeExplainer(model, …)
shap_values = ex.shap_values(X)
shap.force_plot(ex.expected_value, shap_values[john_ind,:], X.iloc[john_ind,:])
Why does 46 years of credit history increase
the risk of payment problems?](https://image.slidesharecdn.com/1140amgpscottlundbergfinalv2-191022184911/75/Scott-Lundberg-Microsoft-Research-Explainable-Machine-Learning-with-Shapley-Values-H2O-World-NYC-2019-13-2048.jpg)
Uploaded bySri Ambati
Scott Lundberg, Microsoft Research - Explainable Machine Learning with Shapley Values - H2O World NYC 2019
The document discusses the use of Shapley values in explainable machine learning, highlighting their importance for understanding and debugging complex models in financial risk assessment and healthcare. It emphasizes the balance between model accuracy and interpretability, the significance of feature attributions, and properties such as additivity and monotonicity. The examples illustrate how Shapley values can help explain predictions and identify potential issues in model performance, ensuring accountability and transparency in AI applications.
More Related Content
Similar to Scott Lundberg, Microsoft Research - Explainable Machine Learning with Shapley Values - H2O World NYC 2019
![[DSC Europe 24] Orsalia Andreou - Fostering Trust in AI-Driven Finance](https://cdn.slidesharecdn.com/ss_thumbnails/orsaliaandreoufosteringtrustinai-drivenfinance-250216113615-6aa23936-thumbnail.jpg?width=640&height=640&fit=bounds)
Scott Lundberg, Microsoft Research - Explainable Machine Learning with Shapley Values - H2O World NYC 2019
- 1. Explainable Machine Learningwith Shapley Values Scott Lundberg Senior Researcher Microsoft
- 2. Explainable AI inpractice Model development
- 3. model 22% chance Johnwill have repayment problems John, a bank customer No loan Why was I denied? What is our financial risk? How do I debug? Accuracy = $
- 4. Interpretable Accurate Complex model✘ ✔ Simple model ✔ ✘ Interpretable or accurate: choose one. 😀 ⚖️ 💰 3 ? ?
- 5. Complex models are inherentlycomplex! But a single prediction involves only a small piece of that complexity. Inputvalue Outputvalue 5
- 6. 6 How did weget here? Base rate Prediction for John 22% 𝑓 𝑥0 16% 𝐸[𝑓 𝑋 ]
- 7. 7 16% 22% 𝐸[𝑓 𝑋] 𝑓 𝑥0 Base rate 𝜙0 Income not verified 18.2% 𝐸 𝑓 𝑋 𝑑𝑜(𝑋1 = 𝑥1)] 𝜙1 No recent account openings 21% 𝐸 𝑓 𝑋 𝑑𝑜(𝑋1,2 = 𝑥1,2)] 22.5% 𝐸 𝑓 𝑋 𝑑𝑜(𝑋1,2,3 = 𝑥1,2,3)] 18.5% 𝐸 𝑓 𝑋 𝑑𝑜(𝑋1,2,3,4 = 𝑥1,2,3,4)] 𝜙2 𝜙3𝜙4 𝜙5 DTI = 30 Delinquent 10 months ago 46 years of credit history The order matters! *Janzing et al. 2019 *
- 8. 8 𝐸[𝑓 𝑋 ]𝑓 𝑥0 𝜙0 𝜙1 No recent account openings 𝜙2 𝜙3 𝜙4 𝜙5 46 years of credit history The order matters! Nobel Prize in 2012 Lloyd Shapley
- 9. 9 𝐸[𝑓 𝑋 ]𝑓 𝑥0 𝜙0 𝜙1 𝜙2 𝜙3 𝜙4 𝜙5 Shapley properties Additivity (local accuracy) – The sum of the local feature attributions equals the difference between rate and the model output. 1
- 10. 10 𝐸[𝑓 𝑋 ]𝑓 𝑥0 𝜙0 𝜙1 𝜙2 𝜙3 𝜙4 𝜙5 Shapley properties Monotonicity (consistency) – If you change the original model such that a feature has a larger possible ordering, then that input’s attribution decrease. 2 Violating consistency means you can’t trust feature orderings based on your attributions. …even within the same model!
- 11. 11 𝐸[𝑓 𝑋 ]𝑓 𝑥0 𝜙0 𝜙1 𝜙2 𝜙3 𝜙4 𝜙5 Shapley values result from averaging over all N! possible orderings. (NP-hard)
- 13. ex = shap.TreeExplainer(model,…) shap_values = ex.shap_values(X) shap.force_plot(ex.expected_value, shap_values[john_ind,:], X.iloc[john_ind,:]) Why does 46 years of credit history increase the risk of payment problems?
- 14. shap.dependence_plot(“Months of credithistory”, shap_values, X) The model is identifying retirement-age individuals based on their long credit histories! Explain and debug your models!
- 15. Explainable AI inpractice Model development Debugging/exploration Monitoring
- 16. Model monitoring Time Training performanceTest performance Can you find where we introduced the bug? 16
- 17. Model monitoring Now canyou find where we introduced the bug? 17 False True
- 18. Model monitoring Time Transient electronicmedical record Time 18 False True
- 19. Model monitoring Time Gradual changein atrial fibrillation ablation procedure durations Time 19 False True
- 20. Explainable AI inpractice Model development Human/AI collaboration Regulatory compliance Debugging/exploration Customer retention Consumer explanations Encoding prior beliefs Monitoring Decision support Anti-discrimination Risk managementHuman risk oversight Scientific discovery Pattern discovery Population subtyping Signal recovery
- 21.
- 22. Global feature importanceLocal explanation summary(A) (log relative risk of mortality) Mortality model (F/M) Mortality risk model
- 23. Global feature importanceLocal explanation summary(A) (log relative risk of mortality) Mortality model (F/M) Mortality risk model
- 24. Global feature importanceLocal explanation summary(A) (log relative risk of mortality) Mortality model (F/M) Mortality risk model
- 25. Global feature importanceLocal explanation summary(A) (log relative risk of mortality) Mortality model (F/M) Mortality risk model
- 26. Global feature importanceLocal explanation summary(A) (log relative risk of mortality) Mortality model (F/M) Mortality risk model
- 27. Global feature importanceLocal explanation summary(A) (log relative risk of mortality) Mortality model (F/M) Mortality risk model
- 28. Reveal rare high-magnitudemortality effects Global feature importance Local explanation summary(A) (log relative risk of mortality) Mortality model (F/M) Conflates the prevalence of an effect with the magnitude of an effect Mortality risk model
- 29. Reveal rare high-magnitudemortality effects Global feature importance Local explanation summary(A) (log relative risk of mortality) Mortality model (F/M)
- 30. Reveal rare high-magnitudemortality effects Global feature importance Local explanation summary(A) (log relative risk of mortality) Mortality model (F/M) Rare high magnitude effects
- 31. Reveal rare high-magnitudemortality effects Global feature importance Local explanation summary(A) (log relative risk of mortality) Mortality model (F/M) Lots of ways to die young…Not many ways to live longer…
Editor's Notes
- #4 We'll start with a simple motivational example. This is John, a typical bank customer. Like many consumers today, when he applies for a loan, information about him is sent through a predictive model XX. This model is designed to calculate the risk that John will have repayment problems, XX which, unfortunately for John, is 55%. And because his risk is high, the bank declines his loan application XX. A natural first question John has XX is “Why?” The bank XX also want to know why, because these are key business decisions. But unfortunately for the data scientist that built this model XX, she used a complex model and so these questions are hard to answer XX …. XX
- #5 The reason the data scientist used a complex model is because they are often very accurate on large data sets XX, but that same complexity also makes them hard to interpret XX. In contrast, simple models are easy to interpret XX, but are often less accurate XX. This leads to a trade-off XX between interpretability and accuracy. This trade-off is particularly painful for the bank since accuracy directly corresponds to profitability XX, while interpretability has important implications for customer satisfaction XX and even legality. This tradeoff effects a wide range of applications, and so many recent methods have been developed to address it. XX
- #6 These methods do not try and make an entire complex model interpretable, because there is inherently too much to succinctly explain. Instead they focus on explaining a single prediction XX, because mapping a single input to an output involves only a small part of the complexity of the overall model. XX
- #7 To see, let’s return to John and answer why his loan application was denied. To explain his denial it is important to start with the base rate of loan repayment problems XX, denoted here by the expected value of the model’s output. To explain John's risk XX, we need to explain how we got from the base rate XX, to his risk of 55%. XX
- #8 To see, let’s return to John and answer why his loan application was denied. To explain his denial it is important to start with the base rate of loan repayment problems XX, denoted here by the expected value of the model’s output. To explain John's risk XX, we need to explain how we got from the base rate XX, to his risk of 55%. XX
- #9 To see, let’s return to John and answer why his loan application was denied. To explain his denial it is important to start with the base rate of loan repayment problems XX, denoted here by the expected value of the model’s output. To explain John's risk XX, we need to explain how we got from the base rate XX, to his risk of 55%. XX
- #10 To see, let’s return to John and answer why his loan application was denied. To explain his denial it is important to start with the base rate of loan repayment problems XX, denoted here by the expected value of the model’s output. To explain John's risk XX, we need to explain how we got from the base rate XX, to his risk of 55%. XX
- #11 To see, let’s return to John and answer why his loan application was denied. To explain his denial it is important to start with the base rate of loan repayment problems XX, denoted here by the expected value of the model’s output. To explain John's risk XX, we need to explain how we got from the base rate XX, to his risk of 55%. XX
- #12 To see, let’s return to John and answer why his loan application was denied. To explain his denial it is important to start with the base rate of loan repayment problems XX, denoted here by the expected value of the model’s output. To explain John's risk XX, we need to explain how we got from the base rate XX, to his risk of 55%. XX
- #17 Make title shorter
- #18 Focus more on for each
- #19 What is an EMR?
- #20 Make title shorter
- #23 Make title shorter
- #24 Make title shorter
- #25 Make title shorter
- #26 Make title shorter
- #27 Make title shorter
- #28 Make title shorter
- #29 Make title shorter
- #30 Make title shorter
- #31 Make title shorter
- #32 Make title shorter