1.
Proposing an Interactive Audit
Pipeline for Visual Privacy
Research
Jasmine DeHart, Chenguang Xu, Lisa Egede, Christan Grant
OUDATALAB.com
2021 IEEE International Conference on Big Data (BigData)
December 15 – 18, 2021

2.
Traditional machine learning pipelines do not consider fairness, privacy,
and ownership issues as they arise.
We recommend frameworks to use for designing new ML pipelines.
In the following slides, we present a scenario that will describe the main
points of the paper.
The Boss Engineer #1 Engineer #2

3.
🪖
You’re now the lead for our
machine learning team.
And, I have this great idea…
You’ve been doing a
great job!
Thanks, Boss!

4.
Let’s build a People
Counter for the downtown
Smart City initiative.
Sounds innovative. I will
put together our
traditional machine
learning pipeline!

5.
There are so many parts
and I have to build this
pipeline from scratch...
I’ll need to design three ML
pipeline phases:
1. Data Preparation Phase;
2. Modeling Phase;
3. Deployment Phase.

6.
Phase 1: Data Preparation
I’ll start with the Data
Preparation Phase.

7.
Phase 2: Modeling
This Modeling Phase
might take a while.

8.
Phase 3: Deployment
This is looking good!

9.
Here is the
complete pipeline
for camera-based
people counter!
Phase 3: Deployment
Phase 1: Data Preparation Phase 2: Modeling

10.
This looks great. It
follows our
traditional pipeline
standards!

11.
We can sell
this model
and data to
companies!
Wait a minute! There are
some additional things
we need to consider.

12.
Historical bias Algorithmic bias Software Discrimination
See paper for more details…
Multiparty Conflict Image Removal Request Obtaining Content
Consent

13.
Human-over-the-loop
• Regular updates help to avoid
and minimize costly errors
• Allows humans to step in pro re
nata to perform corrections or
updates
• Resolve biases that may be
imposed from humans or the
model during learning

14.
Interactive Audit Strategies
Fairness Forensic Auditing System (FASt)
• Inspect a dataset or a model via
techniques and tools for bias
• FASt has three tasks: bias
detection, bias interpretation,
and bias mitigation.
Visual Privacy Auditor (ViP)
• Inspect a dataset or a model via
techniques and tools for privacy
concerns
• Visual privacy mitigation strategies
built into the ViP Auditor.
See paper for more details…

15.
Here’s our updated
pipeline.
Human-over-the-loop
feedback is integrated
with the Audit
Strategies.

16.
Conclusion
• We identify portions of the machine learning pipeline that contain visual
privacy and fairness issues.
• We walkthrough the need for responsible auditing systems to bring
accountability into the ML pipeline.
• We propose using human-over-the-loop strategies for auditing fairness and
privacy issues.

17.
Acknowledgements
• Department of Defense SMART Scholarship
• National Science Foundation Grant # 1952181
• Photo Actors: Makya Stell (The Boss)
Jessica Reese & A’Kile Stone (Engineer 1 & 2)

18.
Backup/Old slides

19.
Intro/Motivate
• Definitions?
• Scenario? Could use reference throughout the presentation

20.
High-level pipeline
• Quick overview of the ml pipeline setup

21.
Issues Pipeline
• Add a issue or two at each spot
• Describe the issues
• Point to the paper

22.
Solution Pipeline
• Discuss those two and how they solve the problem
• FASt
• ViP

23.
Conclude/Future Work