For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2023/10/bias-in-computer-vision-its-bigger-than-facial-recognition-a-presentation-from-santa-clara-university/ Susan Kennedy, Assistant Professor of Philosophy at Santa Clara University, presents the “Bias in Computer Vision—It’s Bigger Than Facial Recognition!” tutorial at the May 2023 Embedded Vision Summit. As AI is increasingly integrated into various industries, concerns about its potential to reproduce or exacerbate bias have become widespread. While the use of AI holds the promise of reducing bias, it can also have unintended consequences, particularly in high-stakes computer vision applications such as facial recognition. However, even seemingly low-stakes computer vision applications such as identifying potholes and damaged roads can also present ethical challenges related to bias. This talk explores how bias in computer vision often poses an ethical challenge, regardless of the stakes involved. Kennedy discusses the limitations of technical solutions aimed at mitigating bias, and why “bias-free” AI may not be achievable. Instead, she focuses on the importance of adopting a “bias-aware” approach to responsible AI design and explores strategies that can be employed to achieve this.

1. Bias in Computer Vision
—It’s Bigger than Facial
Recognition!
Susan Kennedy, PhD
Assistant Professor of Philosophy
Santa Clara University

2. The Optimistic View
2
© 2023 Santa Clara University

3. Bias in Computer Vision: Facial Recognition
3
© 2023 Santa Clara University

4. Bias in CV: Looking Beyond Facial Recognition
4
© 2023 Santa Clara University
Agriculture
Plant Disease Detection
Manufacturing
Quality Inspection
Transportation
Pothole Detection

5. Bias can pose an ethical challenge, even without sensitive data!
Bias in CV: Looking Beyond Facial Recognition
5
© 2023 Santa Clara University
Agriculture
Plant Disease Detection
Manufacturing
Quality Inspection
Transportation
Pothole Detection

6. • Expand the ethical circle to
take into account the full
range of stakeholders
• Who does it not work for?
• What does it not work for?
• When does it not work?
Human Subjects Human Impacts
6
© 2023 Santa Clara University
Core
Direct
Indirect

7. • Toolkits
• Google What-If
• IBM Fairness 360
• Microsoft FairLearn
Mitigating Bias – Technical Solutions
7
© 2023 Santa Clara University

8. • Systematic errors stemming from bias in the
datasets and algorithmic processes used
• Human bias present across the AI lifecycle
and in the use of AI once deployed
• Present in the datasets used in AI, and the
institutional norms and practices across the
AI lifecycle and in broader society
The Tip of the Iceberg
8
© 2023 Santa Clara University

9. • Bias-free AI is an unachievable goal
 Mitigating bias requires a bias-aware approach
• AI exists within a larger social system
 Mitigating bias requires a sociotechnical approach
Bias-Aware not Bias-Free!
9
© 2023 Santa Clara University

10. • Instead of waiting for bias to
strike, build a habit of anticipating
preventable causes so they can be
mitigated
• Across the entire AI lifecycle –
From pre-design to deployment
• 3 key problem areas:
• Datasets
• Testing and evaluation
• Human factors
Responsible AI – Ethical Pre-Mortems
10
© 2023 Santa Clara University

11. Strategies to Employ
11
© 2023 Santa Clara University
• Interrogating decisions about who/what gets counted and how
• Statistical methods to mitigate representation issues
• Culture, context, & stakeholders in terms of dataset suitability
Dataset
• Fairness metrics (context specific!)
• Monitoring performance after deployment
• Periodic model updates, test and recalibrate model parameters
Testing and
Evaluation
• Multistakeholder engagement and diverse perspectives
• Model and procedural transparency
• Algorithmic impact assessments, iterative process
Human
Factors

12. Thermal Imaging for Human-Wildlife Conflict
- Arribada Initiative
12
© 2023 Santa Clara University
Dataset
• Hardware testing – optimizing data
collection methods across species
• Suitability – adjusting for variations in
the environment (dirt, grass, snow)
Human Factors
• Stakeholder engagement – WWF team in
Tezpur advised on locations for field
testing
• Commitment to model transparency

13. 1. Bias poses an ethical challenge, regardless of the
stakes involved
2. Reframing the goal - bias-aware not bias-free
3. An effective mitigation strategy requires a
combination of technical and social considerations
Key Takeaways
13
© 2023 Santa Clara University

14. Resources
14
© 2023 Santa Clara University
1. NIST Special Publication 1270 – Towards a Standard for Identifying and
Managing Bias in Artificial Intelligence
https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.1270.pdf
2. (In Progress) NIST Mitigation of AI/ML Bias in Context
https://www.nccoe.nist.gov/projects/mitigating-aiml-bias-context
3. An Ethical Toolkit for Engineering Design Practice – Markkula Center for
Applied Ethics
https://www.scu.edu/ethics-in-technology-practice/ethical-toolkit/