Designing Fair Models

1. Designing Fair Machine
Learning Models
Charmi Chokshi
ML Engineer | GDE for ML
@CharmiChokshi
Sharmistha Chatterjee
Author | Speaker | GDE for ML
Public Sapient | @sharmichat82

2. Key
Takeaways
-Why Fairness?
-What is Fairness?
-Examples and Demonstration
-Modeling fairness with TensorFlow

3. Let's do an eye test
:P
What do you see?

4. Let's do an eye test
:P
What do you see?
● Bananas
● Bananas on shelves
● Bananas with stickers

5. Let's do an eye test
:P
What do you see?

6. Let's do an eye test
:P
What do you see?
● Green Bananas
● Unripe Bananas

7. Let's do an eye test
:P
What do you see?

8. Let's do an eye test
:P
What do you see?
● Overripe Bananas
● Bananas good for bread

9. Let's do an eye test
:P
What do you see?
● Yellow Bananas
● Yellow is prototypical Bananas!

10. What is
Fairness?

11. What is Fairness?
In machine learning, a given algorithm is said to be fair, or to have
fairness, if
- its results are independent of given variables, especially those
considered sensitive,
- such as the traits of individuals which should not correlate
with the outcome (i.e. gender, ethnicity, sexual orientation,
disability, etc).

12. Does biases affect ML models?

13. 1.Group Attribution Bias
2. Reporting Bias
3. Automation Bias
4. Selection Bias
5. Implicit Bias
Types of Biases

14. How bias in AI
systems affect
various domains

15. 1. AI bias in Healthcare
2. AI bias in Hiring
3. AI bias in Finance
4. AI bias in Law
5. AI bias in Education
6. AI bias in Fashion
Bias in various domains

17. Never trust
speakers, ask
them to show
codes ;)

18. https://github.com/sharmi1206/fairness-tensorflow-
toxicity-classification
https://github.com/sharmi1206/fairness-india-hiring

19. Fairness Problem
Fairer Equitable ML Model across Sensitive Topic Groups
● Dataset - Jigsaw Unintended Bias in Toxicity Classification
● Training, Validation and Test Data set - 902437, 451218, 451218
● Identify small proportion of Sensitive Topics using Fairer Models
● Sensitive topics related to gender, sexuality, religion, race, mental
health, physical health and economic condition

20. TensorFlow Contrained
OptimizationOptimization Techniques
ProxyLagrangianOptimizer with
●Constraints on False Positives
●Constraints on False Positives and
False Negatives
●Non-convex and Non-
Differentiable Constraints
Limitation - No support for multi-
class

21. TensorFlow Contrained
OptimizationRobust Optimization
Different types of Mental Health

22. Results with Robust Optimization
Comparative Study
Smaller Sample
on BERT

23. AI Regulations
The head of Google and
parent company Alphabet has
called for artificial intelligence
(AI) to be regulated
○Fair Marketplace
○Legal Obligation
○Social Responsibility
○Business Requirement

24. Build
Products and
Technologies
that work for
EVERYONE!

25. References
● https://www.kaggle.com/c/jigsaw-unintended-bias-in-toxicity-classification/overview/faq
● https://github.com/google-research/tensorflow_constrained_optimization
● https://github.com/sharmi1206/fairness-tensorflow-toxicity-classification
● https://www.youtube.com/watch?v=6CwzDoE8J4M
● https://indiaai.in/article/how-bias-in-ai-systems-affect-various-domains
● https://developers.google.com/machine-learning/crash-course/fairness/video-lecture
● https://arxiv.org/pdf/1809.04198.pdf

26. Thank You!
Happy Learning
Charmi Chokshi
ML Engineer | GDE for ML
Sharmistha Chatterjee
Author | Speaker | GDE for ML

27. Questions?
Suggestions?
Comments?
Charmi Chokshi
ML Engineer | GDE for ML
Sharmistha Chatterjee
Author | Speaker | GDE for ML