We now live in a world where we trust intelligent systems blindly, believing in their rationality and objectivity. However, in reality this is far from the truth. In this talk given at the City.AI Singapore chapter, we explored the nature, implications and handling strategies for Model Bias in AI.

1. MODEL BIAS IN AI
17 Aug 2017
Jason Tamara Widjaja

2. A biased sample size of one
I am:
• Made in Singapore, 8 years overseas
• Plugging into deep tech: TNB Ventures and
Lauretta.io
I work:
• Data science and AI at MSD
• IT, IS, HR, MBA, Analytics, AI
I like:
• Building world class data science and AI teams
• “Tell me like I’m five” accessible communication

3. Agenda
3
1 Introduction: What Is AI in MSD?
2 The Problem: Model Bias in Practice
3 The Solution: A Framework for Mitigating Model Bias
4 Looking Ahead: Towards A Safer AI-enabled Future

4. Acting Humanly
“Turing Test”
Making sense of the AI space
4
Thinking Humanly
“Cognitive modeling”
Thinking Rationally
“Logical laws of thought”
Acting Rationally
“Intelligent agents”

5. Acting Humanly
“Turing Test”
Making sense of the AI space
5
Thinking Humanly
“Cognitive modeling”
Thinking Rationally
“Logical laws of thought”
Acting Rationally
“Intelligent agents”

6. Descriptive
analytics
Predictive
analytics
Prescriptive
analytics
AI as an extension of data science focused on replicating the capabilities of intelligent agents
Human
agency
Decision support
“Tool”
Back office
automation / RPA
Human interaction
automation
Intelligent agent
“Worker”
Autonomous
execution

7. Agenda
7
1 Introduction: AI in MSD
2 The Problem: Model Bias in Practice
3 Fixing Projects: Pressure Points
4 Fixing Systems: Broader Considerations
1 Introduction: What Is AI in MSD?
2 The Problem: Model Bias in Practice
3 The Solution: A Framework for Mitigating Model Bias
4 Looking Ahead: Towards A Safer AI-enabled Future

8. A thought experiment: What does being on the receiving end of a model feel like?
“How can I assign my people to different training programs?”
40,000 people
4 categories of 100 features each
4 tiers of performance: low, medium, high, v.high
Different development tracks
Data collection
Supervised learning
Clustering within each tier

9. A thought experiment: What does being on the receiving end of a model feel like?
“How can I assign my people to different training programs?”
40,000 people
4 categories of 100 features each
Different development tracks
Data collection
Supervised learning
Clustering within each tier
4 tiers of performance: low, medium, high, v.high

10. A thought experiment: What does being on the receiving end of a model feel like?
“How can I assign my people to different training programs?”
40,000 people
4 categories of 100 features each
4 tiers: low, medium, high, talent
Different development tracks
Data collection
Supervised learning
Clustering within each tier
Big questions:
• Did the model favour one ‘group’ over
another?
• Did the model use information besides real
performance in its decision?
• Is the model “fair”?

11. 11
Now what if the model was fully automated?
No appeals. No explanations. No human intervention.

12. Multiple studies have found evidence that automated, opaque models systemically
disadvantage certain demographic groups at scale

13. Google’s search results for ‘CEO’ – 14 August 2017

14. If we deploy models trained on existing, biased data sources, we may be unknowingly
perpetuating discrimination

15. 15
Models do not automatically remove bias - used carelessly they may
systematize it

16. Agenda
16
1 Introduction: AI in MSD
2 The Problem: Model Bias in Practice
3 Fixing Projects: Pressure Points
4 Fixing Systems: Broader Considerations
1 Introduction: What Is AI in MSD?
2 The Problem: Model Bias in Practice
3 The Solution: A Framework for Mitigating Model Bias
4 Looking Ahead: Towards A Safer AI-enabled Future

17. A 4-part framework for mitigating model bias
2. MODEL
INTERPRETABILITY
1. DEVELOPER
CHOICE
3. MANAGEMENT
POLICY
4. USER
INTERPRETATION
More transparent tool
“Make the black box less black”
More skillful users
“Make black box users more proficient”

18. A 4-part framework for mitigating model bias
2. MODEL
INTERPRETABILITY
1. DEVELOPER
CHOICE
3. MANAGEMENT
POLICY
4. USER
INTERPRETATION
More transparent tool
“Make the black box less black”
More skillful users
“Make black box users more proficient”

19. A 4-part framework for mitigating model bias
MODEL
INTERPRETABILITY
DEVELOPER
CHOICE
MANAGEMENT
POLICY
USER
INTERPRETATION
“Build models thoughtfully for safer, more understandable models”
Choose your features consciously

20. A 4-part framework for mitigating model bias
Choose your features consciously Trade predictive power for more interpretable models
“Build models thoughtfully for safer, more understandable models”
MODEL
INTERPRETABILITY
DEVELOPER
CHOICE
MANAGEMENT
POLICY
USER
INTERPRETATION

21. A 4-part framework for mitigating model bias
“Add forensics to models to build trust”
QII for understandable features
(Quantitative Input Influence)
LIME for non-understandable features
(Local Interpretable Model-agnostic
Explanations)
MODEL
INTERPRETABILITY
DEVELOPER
CHOICE
MANAGEMENT
POLICY
USER
INTERPRETATION
Key lime (QII – LIME) pie

22. A 4-part framework for mitigating model bias
QII for understandable features
MODEL
INTERPRETABILITY
DEVELOPER
CHOICE
MANAGEMENT
POLICY
USER
INTERPRETATION
Personal Transparency Reports
Why was person X classified this way?
Group Impact Diagnostics
When I add feature X, what changes?
Model Diagnostics
Which features drive model X?
“Add forensics to models to build trust”

23. A 4-part framework for mitigating model bias
LIME for non-understandable features: GOOD MODEL
“Add forensics after modeling so approaches are less constrained”
MODEL
INTERPRETABILITY
DEVELOPER
CHOICE
MANAGEMENT
POLICY
USER
INTERPRETATION

24. A 4-part framework for mitigating model bias
LIME for non-understandable features: BAD MODEL
“Add forensics after modeling so approaches are less constrained”
MODEL
INTERPRETABILITY
DEVELOPER
CHOICE
MANAGEMENT
POLICY
USER
INTERPRETATION

25. A 4-part framework for mitigating model bias
2. MODEL
INTERPRETABILITY
1. DEVELOPER
CHOICE
3. MANAGEMENT
POLICY
4. USER
INTERPRETATION
More transparent tool
“Make the black box less black”
More skillful users
“Make black box users more proficient”

26. A 4-part framework for mitigating model bias
“Have a management conversation – what is a fair model?
Optimise for more than ROI”
Demographic
Parity
X% of men get a chance
X% of women get a chance
Equal
Opportunity
X% of qualified men get a chance
X% of qualified women get a chance
MODEL
INTERPRETABILITY
DEVELOPER
CHOICE
MANAGEMENT
POLICY
USER
INTERPRETATION

27. A 4-part framework for mitigating model bias
“Train users of AI systems in model diagnostics and AI quality assurance”
MODEL
INTERPRETABILITY
DEVELOPER
CHOICE
MANAGEMENT
POLICY
USER
INTERPRETATION

28. Agenda
28
1 Introduction: AI in MSD
2 The Problem: Model Bias in Practice
3 Fixing Projects: Pressure Points
4 Fixing Systems: Broader Considerations
1 Introduction: What Is AI in MSD?
2 The Problem: Model Bias in Practice
3 The Solution: A Framework for Mitigating Model Bias
4 Looking Ahead: Towards A Safer AI-enabled Future

29. 5 VALUES FROM THE ASILOMAR AI PRINCIPLES
7) Failure Transparency: If an AI system causes harm, it should be possible to ascertain why.
9) Responsibility: Designers and builders of advanced AI systems are stakeholders in the moral implications of
their use, misuse, and actions, with a responsibility and opportunity to shape those implications.
10) Value Alignment: Highly autonomous AI systems should be designed so that their goals and behaviors can
be assured to align with human values throughout their operation.
11) Human Values: AI systems should be designed and operated so as to be compatible with ideals of human
dignity, rights, freedoms, and cultural diversity.
15) Shared Prosperity: The economic prosperity created by AI should be shared broadly, to benefit all of
humanity.
Jaan Tallinn, Co-
founder, Skype
Viktoriya Krakovna, AI
Safety, DeepMind
Nick Bostrom,
Director, Oxford
Future of Humanity
Institute
Erik Brynjolfsson,
Director, MIT Center
for Digital Business,
MIT
Stephen Hawking,
Director of Research,
Centre for Theoretical
Cosmology,
Cambridge University
Stuart Russell,
Professor of AI, UC
Berkeley
Elon Musk, Founder,
SpaceX and Tesla
Motors

30. Appendix and references
1. Ribeiro, M. T., Singh, S., & Guestrin, C. (2016, August). Why should I trust you?: Explaining the predictions of
any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery
and Data Mining (pp. 1135-1144). ACM.
2. Datta, A., Sen, S., & Zick, Y. (2017). Algorithmic transparency via quantitative input influence. In Transparent
Data Mining for Big and Small Data (pp. 71-94). Springer International Publishing.
3. Hardt, M., Price, E., & Srebro, N. (2016). Equality of opportunity in supervised learning. In Advances in
Neural Information Processing Systems (pp. 3315-3323).
4. Gunning, D. (2017). Explainable artificial intelligence (xai). Defense Advanced Research Projects Agency
(DARPA), nd Web.
5. AI Principles. (n.d.). Retrieved August 10, 2017, from https://futureoflife.org/ai-principles/
30

31. 31
Our tools need to be sharper, but they also need to be safer
Industry 4.0 needs to grow up alongside ethics 4.0