“Combating Bias in Production Computer Vision Systems,” a Presentation from Red Cell Partners
•
0 likes•6 views
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/combating-bias-in-production-computer-vision-systems-a-presentation-from-red-cell-partners/ Alex Thaman, Chief Architect at Red Cell Partners, presents the “Combating Bias in Production Computer Vision Systems” tutorial at the May 2023 Embedded Vision Summit. Bias is a critical challenge in predictive and generative AI that involves images of humans. People have a variety of body shapes, skin tones and other features that can be challenging to represent completely in training data. Without attention to bias risks, ML systems have the potential to treat people unfairly, and even to make humans more likely to do so. In this talk, Thaman examines the ways in which bias can arise in visual AI systems. He shares techniques for detecting bias and strategies for minimizing it in production AI systems.
Recommended
Recommended
More Related Content
Similar to “Combating Bias in Production Computer Vision Systems,” a Presentation from Red Cell Partners
Similar to “Combating Bias in Production Computer Vision Systems,” a Presentation from Red Cell Partners (20)
More from Edge AI and Vision Alliance
More from Edge AI and Vision Alliance (20)
Recently uploaded
Recently uploaded (20)
“Combating Bias in Production Computer Vision Systems,” a Presentation from Red Cell Partners
- 1. Combating Bias in Production Computer Vision Systems Alex Thaman Chief Architect Red Cell Partners
- 2. About Red Cell Partners 2 © 2023 Red Cell Partners Red Cell is an incubation firm that builds & invests in rapidly scalable, technology-led companies bringing revolutionary advancements to market in National Security & Healthcare, sectors where we have a distinct competitive advantage. Our Mission United by a shared sense of duty & deep belief in the power of innovation, we develop technology to address our Nation’s most pressing problems. Domain knowledge Technology expertise (Big Data, AI/ML, AR/VR, Kinetics, etc.) Strategic networks Exceptional capabilities in regulated end markets Our Strategy We incubate & invest in technology-led companies that address key issues in National Security & Healthcare, backing the most promising emerging firms & dramatically accelerating their growth by leveraging our team’s:
- 3. 3 © 2023 Red Cell Partners What Is The Problem?
- 4. Understanding and Evaluating Racial Biases in Image Captioning • Males appear 2x more than females • Light skin appears 7.5x more than dark • Dark-skinned females appear 23.1x less frequently than dark-skinned males • Lighter-skinned appear more with indoor and furniture objects • Darker-skinned appear more with outdoor and vehicle objects Dataset Statistics in Coco Dataset 4 © 2023 Red Cell Partners Zhao, D., Wang, A., & Russakovsky, O. (2021). Understanding and Evaluating Racial Biases in Image Captioning. In International Conference on Computer Vision (ICCV). 1 2 3 4 5 6
- 5. • CLIP • 400M images from 500K word queries based based on frequency in Wikipedia • Multimodal neurons in artificial neural networks (openai.com) • We have observed, for example, a “Middle East” neuron with an association with terrorism; and an “immigration” neuron that responds to Latin America. We have even found a neuron that fires for both dark-skinned people and gorillas... Encoded Bias 5 © 2023 Red Cell Partners https://microscope.openai.com/models/contrastive_v2/image_ block_4_2_Add_6_0/1895
- 6. Generative AI Bias 6 © 2023 Red Cell Partners Prompt: “A person cleaning a living room” Source: Bing Image Creator
- 7. Generative AI Bias 7 © 2023 Red Cell Partners Prompt: “A housecleaner cleaning a bathroom” Source: Bing Image Creator
- 8. Generative AI Bias 8 © 2023 Red Cell Partners Search Query: “A housecleaner cleaning a bathroom” Source: Google Image Search
- 9. Generative AI Bias 9 © 2023 Red Cell Partners Prompt: “A nurse taking care of a hospital patient” Source: Bing Image Creator
- 10. 10 © 2023 Red Cell Partners Measuring Bias
- 11. • Labeling data for analytics • Distribution mismatches • Find correlations • Consider labeling costs • Sampling your samples • Instructions • Pre-labeling Metadata Labeling 11 © 2023 Red Cell Partners Zhao, D., Wang, A., & Russakovsky, O. (2021). Understanding and Evaluating Racial Biases in Image Captioning. In International Conference on Computer Vision (ICCV).
- 12. • Fitzpatrick scale • 6 classifications • Skewed towards white skin variations • Monk scale • 10 classifications • Tested socially across population groups for fair representation Skin Tone Labeling 12 © 2023 Red Cell Partners
- 13. Metadata Statistics 13 © 2023 Red Cell Partners
- 14. Metadata Statistics 14 © 2023 Red Cell Partners
- 15. • Live data collection • Sample data in production • Compare inference results across dimensions • Without access to the data • Human supervision is difficult • Use metadata models for insights • Data drift in embedding space – KL divergence, WS distance, etc. Measuring Bias in Production 15 © 2023 Red Cell Partners UMAP of CLIP embedding of cropped boxes, colored by class
- 16. 16 © 2023 Red Cell Partners Controlling Bias
- 17. • Data collection • Targeted data collection for specific population groups • Collect more data than you think you need • Dataset balancing • Sample training data across dimensions • Overweight underrepresented data Controlling Bias 17 © 2023 Red Cell Partners
- 18. • Synthetic data • Infinite labels/metadata • Study model behavior across dimensions • Learn domain-invariant features via pre-training Controlling Bias 18 © 2023 Red Cell Partners Microsoft Face Synthetics: https://microsoft.github.io/FaceSynthetics/
- 19. Synthetic Data Study 19 © 2023 Red Cell Partners
- 20. 20 © 2023 Red Cell Partners Other Considerations
- 21. • Skin tone • Perceived differently by different cultures • Multidimensional • Image characteristics and environment • Sex/gender not globally normalized Descriptive Metadata Challenges 21 © 2023 Red Cell Partners What skin tone are these 2 people?
- 22. • “To have any inductive process make predictions on unseen data, an agent requires a bias. What constitutes a good bias is an empirical question about which biases work best in practice” Poole, D. & Mackworth, A. (2019). Artificial Intelligence: Foundations of Computational Agents. • Think about tradeoffs • Can the application layer help? The Cost of Bias 22 © 2023 Red Cell Partners Clothes fitting Racial associations Obscure Misrecognitions Helpful Harmful
- 23. • All AI systems have bias that results from the data that was used • AI systems need deep evaluation prior to widespread deployment • Analysis through additional metadata, statistics, and controlled experiments help predict how these systems may perform in the real world Conclusions 23 © 2023 Red Cell Partners
- 24. Resources Contact Info LinkedIn: https://www.linkedin.com/in/alex- thaman-93436659/ Email: alex.thaman@redcellpartners.com 24 © 2023 Red Cell Partners GitHub repository for this talk https://github.com/alexthaman/evs2023 Understanding and Evaluating Racial Biases in Image Captioning https://arxiv.org/abs/2106.08503 Themis AI – evaluate bias in your model https://themisai.io/
- 25. 25 © 2023 Red Cell Partners Appendix
- 26. • Detecting data drift - https://www.evidentlyai.com/blog/data-drift- detection-large-datasets • Monk Scale blog https://blog.google/technology/research/ai-monk-scale-skin-tone- story/ • REVISE (open source tool for evaluating bias) - https://github.com/princetonvisualai/revise-tool • Other products to evaluate bias: Manot (https://www.manot.ai/) Additional Resources 26 © 2023 Red Cell Partners
- 27. • Digital Humans Synthetic data • Synthesis AI - https://synthesis.ai/ • Datagen - https://datagen.tech/ • Infinity AI - https://infinity.ai/ • Unity Digital Humans - https://github.com/Unity- Technologies/com.unity.cv.synthetichumans • Microsoft Face Synthetics - https://microsoft.github.io/FaceSynthetics/ Additional Resources 27 © 2023 Red Cell Partners
- 28. Bias AI systems produce biased humans: Experiment Setup AI-Human Feedback Loop 28 © 2023 Red Cell Partners Glickman, M., & Sharot, T. (2022, November 15). Biased AI systems produce biased humans. https://doi.org/10.31219/osf.io/c4e7r
- 29. AI-Human Feedback Loop 29 © 2023 Red Cell Partners Glickman, M., & Sharot, T. (2022, November 15). Biased AI systems produce biased humans. https://doi.org/10.31219/osf.io/c4e7r Conclusion 1: Human-AI interactions create bias feedback loops
- 30. AI-Human Feedback Loop 30 © 2023 Red Cell Partners Glickman, M., & Sharot, T. (2022, November 15). Biased AI systems produce biased humans. https://doi.org/10.31219/osf.io/c4e7r Conclusion 2: Humans underestimate the impact of the bias from AI