Recommended
More Related Content
Similar to When the AIs failures send us back to our own societal biases
Similar to When the AIs failures send us back to our own societal biases (20)
Recently uploaded
Recently uploaded (20)
When the AIs failures send us back to our own societal biases
- 2. 2 The good and the evil
- 3. 3 The good and the evil
- 4. 4 Mélanie DUCOFFE PhD., Research Data Scientist @mducoffe melanie.ducoffe@airbus.com Deep Learning, AI explainability About me !
- 5. 5 Clément DUFFAU PhD., Lead DevOps @clement0210 clement.duffau@stack-labs.com Automation, verification & validation, assurance quality, safety About me !
- 6. 6 Summary Artificial Intelligence and ethics Data selection biases In production biases Train interpretation biases
- 7. 7 Job posts gathered into a single position: prone to many biases. Data biases and Model biases What do you think of combining : architect + dev+ testing + maintenance ? The nature of AI into the work market Data Scientist Data Analyst Data Engineer Machine Learning Scientist Machine Learning Engineer Statistician Find, clean and organize data for companies Transform and manipulate large datasets to suit the analysis of companies Perform batch processing or real time processing on gathered/stored data Research new data approaches and algorithms Create data funnels and deliver software solutions Interpret, analyze and report statistical information
- 8. 8 ● Bias (in statistics) : “the difference between the expectation of the sample estimator and the true population value, which reduces the representativeness of the estimator by systematically distorting it” ● “The big takeaway is that we don’t know what we don’t know,” (Alice Popejoy) What do you mean about biases ?
- 10. 10 Ethics in AI
- 11. 11 ● MIT study decision-making of self-driving car on killing scenario ● Scenario with ○ high/low level of education ○ young/old ○ male/female ○ pets ○ traffic signal respect http://moralmachine.mit.edu/ The Moral Machine
- 12. 12 ● Results “rank” you on ○ number of death ○ law respect ○ gender,age,health, ... ● Need ethical choices at the government, insurance ? manufacturer ? passengers ? The Moral Machine
- 13. 13 ● 3 different clusters ● Cultural differences ○ USA/Europe kill the oldest vs Japon saves the oldest ○ Colombia saves high educated people vs Finland doesn’t matter ○ South america/France saves women The Moral Machine experiment, Edmond Awad et al. (2018), Nature The Moral Machine
- 14. 14 Ethics in clinical investigations ● Technical committee to review scientific foundations and safety (in France, ANSM) ● Ethics committee on animal AND humans (in France, CPP) ○ Composed by medical professional and citizen ○ Review application form on ■ benefit/risk ■ information quality ■ resources to conduct the study ■ patient recruitment process ■ Patient consent modality ○ Follow the experiment process
- 15. 15 Applying ethics committee in AI at Google ● Advanced Technology External Advisory Council (ATEAC) ○ Ensure the white paper principles for AI at Google ■ Be socially beneficial ■ Avoid creating or reinforcing unfair bias ■ Be built and tested for safety ■ Be accountable to people ■ Incorporate privacy design principles ■ Uphold high standards of scientific excellence ■ Be made available for uses that accord with these principles ● Dissolved 1 week after creation ○ Ethics of some attendees were discussed … ○ Questioning themselves of the needs to represent every part of the society
- 17. 17 ● In the 90s, “I was breaking all of [my classmates’] facial-recognition software because apparently all the pictures they were taking were of people with significantly less melanin than I have” (Charles Isbell) ● In 2015, the “gorilla mistake” in Google Photos ● 25 years, not the same learning model at all, but the same root cause Face recognition from the 90s to nowadays
- 18. 18 ● In 2014, Amazon develops an IA to “find” key success factor and hire these people ● Train on their 10 last years hired people ● 89% of the engineering workforce is male ● At the end, you have sexist recruitment AI ! ● Representative data inside the company by not inside the society … AI recruitment by Amazon
- 19. 19 Biases in the Features Outcome outputFeatures Model Diploma Gender Hobbies Nationality University ... HIRED ? Model Outcome output Features ● Lack of an appropriate set of features ● Lack of an appropriate dataset ● Imbalanced dataset or bias in the output ● Unawareness: remove sensitive features from your data
- 20. 20 Tutorial: your first bias detector Survival of passengers on the Titanic ● Decision Tree ● Leaves = class labels ● Nodes = splitting/conjunctions of features ● Important nodes = less deep, lot of observations ● Many Framework coexist: FairML, “What If”, IBM Bias Assessment ● ‘No universal solution’: combine them Summarize: Your chance of survival were good iff: - you were a woman - you were a young boy with few siblings
- 21. 21 ● Bad advice pointed out by IBM in internal in 2017 ○ ex : Suggest a cancer patient with severe bleeding be given a drug that could cause the bleeding to worsen ● Started off by using real patient data ● Fed it with hypothetical data ● “Synthetic cases allow you to treat and train Watson on a variety of patient variables and conditions that might not be present in random patient samples, but are important to treatment recommendations” (Edward Barbani) ● Pointed out the difficulties to collect representative data in medical Unsafe medical recommendations by IBM Watson
- 22. 22 Data Augmentation ● Historically for images: rotation, flipping, adding noise… ● Object detection models : performance loss on corrupted images ● CNNs generalize poorly to novel distortion types, despite being trained on a variety of other distortions
- 23. 23 Generating new data with GANS source: Medium, “Generative Adversarial with Maths” by Madhu Sanjeevi Discriminator Generator Generator
- 24. 24 Understanding biases and Generalization in Deep Generative Models Can we learn the distribution of data with GANS ? What are the inductive biases of Deep Generative models ? ● Unbiased and consistent density estimation impossible ● Inductive biases ● Similar cognitive bias as humans: numerosity ● Weber’s law: relative change (ratio) True data Generated data
- 25. 25 GAN-based Data Augmentation Perpetuates biases
- 26. 26
- 28. 28 Group Fairness ● Demographic Parity ○ ex : gender independent ● Equal odds ○ ex : take into account the reality statistics in one side ● Equal opportunity ○ ex : take into account the reality statistics in both side
- 31. 31 Black box models: non linear decision functions DOG CAT WHY
- 32. 32 LIME: Zooming ! create neighbors and deduce a linear classifier DOG CAT DOG CAT
- 33. 33 Is it working ? Examples of Explanations are usually cherry-picked !
- 34. 34 Individual Fairness for Explainability The fact that 2 resulting saliency maps are different is fundamentally due to the network itself being fragile to such perturbations
- 35. 35 Autonomous Vehicle Visualization at Uber
- 37. 37 ● Experienced lane recognition algorithm ● Doesn’t expect that 3 stickers break the robustness ● Hacking the AI with thinks that doesn’t matter humans Tesla autonomous driving hacked with stickers
- 39. 39 Hacking tutorial: design your own adversarial example CAT DOG CAT DOG if B is predicted as a cat, retry to cross the decision boundary with B ! Linear Model NON Linear Model A A B B
- 40. 40 Hacking Tutorial: I don’t have access to the model This is not a car ! This is not a car !
- 41. 41 Defending against adversarial attacks ?! SUCCESS FAILURES 2013 Discovery of adversarial examples 2015 Fast Gradient Sign 2015 DeepFool 2016 Carlini Wagner 2019 Unforeseen Attack 2013 Adversarial Training BRUTE FORCE: training with adv 2015 Defensive Distillation output probabilities rather than hard decisions 2016 JPEG compression against adv
- 42. 42 Why is it hard to defend against adversarial examples? ● Adaptative: block one type of attacks but leaves vulnerability open to an attacker who knows the defense being used ● Hard to defend because hard to understand the theory ?
- 43. 43 Adversarial Examples are not Bugs, they are features BIRD DOG CAT “Bird” “Dog” “Cat” create the label-target adversarial dataset DOG CAT BIRD “Dog” “Cat” “Bird”
- 45. 45 Conclusion Data Biases Model Biases Explanation Prod Biases Adv Examples If no one cares, it is highly likely that the next person who suffers from biased treatment is one of us
- 46. 46 What about in the future ? ● Art project exposed racial bias in the biggest image dataset ImageNet in Sept. 2019 ● ImageNet will remove 600,000 images ● Number of publications per year with the name Imagenet in the abstract…
- 47. 47 THANKS!