Proposing an Interactive Audit Pipeline for Visual Privacy Research
1. Proposing an Interactive Audit
Pipeline for Visual Privacy
Research
Jasmine DeHart, Chenguang Xu, Lisa Egede, Christan Grant
OUDATALAB.com
2021 IEEE International Conference on Big Data (BigData)
December 15 – 18, 2021
2. Traditional machine learning pipelines do not consider fairness, privacy,
and ownership issues as they arise.
We recommend frameworks to use for designing new ML pipelines.
In the following slides, we present a scenario that will describe the main
points of the paper.
The Boss Engineer #1 Engineer #2
3. 🪖
You’re now the lead for our
machine learning team.
And, I have this great idea…
You’ve been doing a
great job!
Thanks, Boss!
4. Let’s build a People
Counter for the downtown
Smart City initiative.
Sounds innovative. I will
put together our
traditional machine
learning pipeline!
5. There are so many parts
and I have to build this
pipeline from scratch...
I’ll need to design three ML
pipeline phases:
1. Data Preparation Phase;
2. Modeling Phase;
3. Deployment Phase.
6. Phase 1: Data Preparation
I’ll start with the Data
Preparation Phase.
11. We can sell
this model
and data to
companies!
Wait a minute! There are
some additional things
we need to consider.
12. Historical bias Algorithmic bias Software Discrimination
See paper for more details…
Multiparty Conflict Image Removal Request Obtaining Content
Consent
13. Human-over-the-loop
• Regular updates help to avoid
and minimize costly errors
• Allows humans to step in pro re
nata to perform corrections or
updates
• Resolve biases that may be
imposed from humans or the
model during learning
14. Interactive Audit Strategies
Fairness Forensic Auditing System (FASt)
• Inspect a dataset or a model via
techniques and tools for bias
• FASt has three tasks: bias
detection, bias interpretation,
and bias mitigation.
Visual Privacy Auditor (ViP)
• Inspect a dataset or a model via
techniques and tools for privacy
concerns
• Visual privacy mitigation strategies
built into the ViP Auditor.
See paper for more details…
16. Conclusion
• We identify portions of the machine learning pipeline that contain visual
privacy and fairness issues.
• We walkthrough the need for responsible auditing systems to bring
accountability into the ML pipeline.
• We propose using human-over-the-loop strategies for auditing fairness and
privacy issues.
17. Acknowledgements
• Department of Defense SMART Scholarship
• National Science Foundation Grant # 1952181
• Photo Actors: Makya Stell (The Boss)
Jessica Reese & A’Kile Stone (Engineer 1 & 2)