4. Outline
● What happening now?
● What is interpretability and why do we need it?
● How to interpret (a) model(s)?
● Why cares about it?
● What tools I can use?
● References
● Takeaways
32. Overall, Northpointe’s assessment tool correctly predicts recidivism
61 percent of the time. But blacks are almost twice as likely as whites
to be labeled a higher risk but not actually re-offend. It makes the
opposite mistake among whites: They are much more likely than
blacks to be labeled lower risk but go on to commit other crimes.
https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
59. “Interpretability is the degree to which a
human can
understand the cause of a decision.”
Tim Milner on “Explanation in Artificial Intelligence:
Insights from the Social Sciences”
https://arxiv.org/abs/1706.07269
63. 5 Reasons for Interpretability
Compliance to
legislation
Verify that classifier works as
expected
Interpretability in
science
Learn from the learning
machine
Improve Classifier
85. IML package for R
Features included:
● Feature importance
● Partial dependence plots
● Individual conditional
expectation plots (ICE)
● Accumulated local effects
● Tree surrogate
● LocalModel: Local Interpretable
Model-agnostic Explanations
● Shapley value for explaining
single predictions
https://github.com/christophM/iml
91. Yellowbrick
Python Package
Features included:
● Feature Visualization
● Classification Visualization
● Model Selection Visualization
● Target Visualization
● Text Visualization
https://www.scikit-yb.org/en/latest/
92.
93. Skater Python Package
Feature includes:
● Global Interpretation
○ Feature Importance
○ Partial Dependence Plot
● Local Interpretation
○ LIME
○ Layer-Wise Relevance
Propagation
○ Integrated Gradient
● Hybrid
○ Scalable Bayesian Rule List
○ Tree Surrogateshttps://oracle.github.io/Skater/overview.html
94. ELI5
Python Package
Features included:
● Explainer for model built by
○ Scikit-Learn
○ Keras
○ XGBoost
○ LightGBM
○ CatBoost
○ Sklearn-CRFSuite
● Permutation Importance
● Text Explainer
https://eli5.readthedocs.io/en/latest/
95. Aequitas Python Package for Model
Auditing
Metrics Included:
● Predicted Positive
● Total Predictive Positive
● Predicted Prevalence
● False Discovery Rate
● False Omission Rate
● Visualizationshttp://www.datasciencepublicpolicy.org/projects/aequitas/