Presented at All Things Open 2023 Presented by Christine Abernathy - F5, Inc. Title: Fortifying the Future: Tackling Security Challenges in AI/ML Applications Abstract: As Artificial Intelligence (AI) and Machine Learning (ML) applications continue to surge, it is crucial to be aware of and address the security risks associated with these technologies. In this talk, Christine will explore AI/ML failure modes, threats, and mitigation strategies. She will guide you through the fundamentals of ML models then introduce you to key security challenges such as adversarial attacks, data poisoning, model inversion, model stealing, and membership inference attacks, using real-world examples to demonstrate their potential impact. Christine will also discuss privacy and ethical considerations in ML, touching upon techniques like federated learning and shedding light on the current regulatory landscape surrounding security risks. If you are developing AI/ML applications or incorporating AI/ML components into your technology stack, check out this talk. You will walk away with a deeper understanding of the current AI/ML security landscape and a toolkit to help you address these risks, enabling you to build safer, more secure, and privacy-aware applications. Find more info about All Things Open: On the web: https://www.allthingsopen.org/ Twitter: https://twitter.com/AllThingsOpen LinkedIn: https://www.linkedin.com/company/all-things-open/ Instagram: https://www.instagram.com/allthingsopen/ Facebook: https://www.facebook.com/AllThingsOpen Mastodon: https://mastodon.social/@allthingsopen Threads: https://www.threads.net/@allthingsopen 2023 conference: https://2023.allthingsopen.org/

1. Fortifying the Future:
Tackling Security Challenges
in AI/ML Applications
All Things Open, October 2023
Christine Abernathy (she/her)
@abernathyca

2. ©2023 F5
2
Have you
tried
ChatGPT?
Artist in residence thanks
to Stable Diffusion :)
Yay Code LLaMa!
Learning Python this
weekend.
Mistral 7B
what!!
Prompt Engineer needed with at
least 5 years experience

3. ©2023 F5
3
What could go wrong…

4. ©2023 F5
4
What could go wrong…

5. ©2023 F5
5
What is Machine Learning?

6. ©2023 F5
6
Train machines to
perform certain tasks
Mimic human actions
and thoughts
Handle any cognitive task at
human+ levels

7. ©2023 F5
7
Supervised Learning
Unsupervised Learning
Reinforcement Learning

8. ©2023 F5
8
Deep Learning

9. ©2023 F5
9
Llama
> Supervised Learning
Deep Learning

10. ©2023 F5
10
What could go wrong…

11. ©2023 F5
11
What could go wrong…
During Model Training After Model Trained After Model Deployed
Spider
“Learning the wrong thing” “Revealing the wrong thing” “Doing the wrong thing”
Integrity Confidentiality Integrity

12. ©2023 F5
12
Reference: https://arxiv.org/pdf/1904.08653.pdf Reference: https://arxiv.org/pdf/1712.09665.pdf
ML Attacks > Adversarial Inputs
Prediction
Noise
Reference: https://arxiv.org/pdf/1707.08945.pdf

13. ©2022 F5
13
• Adversarial Training
• Input Sanitization
• Defensive Distillation
• Gradient Masking / Shielding
Reference: https://arxiv.org/pdf/1904.08653.pdf Reference: https://arxiv.org/pdf/1712.09665.pdf
Prediction
Noise
Reference: https://arxiv.org/pdf/1707.08945.pdf
ML Attacks > Adversarial Inputs
Countermeasures

14. ©2023 F5
14
ML Attacks > Model Inversion
• Limit Model Access
• Regularization
• Differential Privacy
Countermeasures
Prediction:
- Name: Null
- Home: Null
- Confidence: 0
Prediction:
- Name: Baby Yoda
- Home: Coruscant
- Confidence: 0.61
Prediction:
- Name: Baby Yoda
- Home: Coruscant
- Confidence: 0.93

15. ©2023 F5
15
ML Attacks > Membership Inference
• Differential Privacy
• Regularization
• Model Generalization
Countermeasures
Prediction:
- Name: Baby Groot
- Home: Planet X
- Confidence: 0.95
Prediction:
- Name: Baby Yoda
- Home: Coruscant
- Confidence: 0.93
Prediction:
- Name: Mac Jack
- Home: Earth
- Confidence: 0.60

16. ©2023 F5
16
ML Attacks > Model Stealing
• Limit Model Access
• API Protection
• Output Perturbation
Countermeasures
Reference: https://nvd.nist.gov/vuln/detail/CVE-2019-20634

17. ©2023 F5
17
ML Attacks > Data Poisoning
• Data Sanitization
• Outlier Detection
• Model Verification
Countermeasures

18. ©2023 F5
18
ML Attacks > Common Mitigation Strategies
• Monitoring and Logging
• Access Control
• Model Updates

19. ©2023 F5
19
Are we secure now?

20. ©2023 F5
20
Source: https://ml-ops.org/content/end-to-end-ml-workflow

21. ©2023 F5
21
Development
Requirements Design Testing Deployment
Maintenance,
Monitoring
Shift Left
Threat
modeling
Secure coding
Static
analysis
Incidence
response plan
Vulnerability
patches
Risk
assessments

22. ©2023 F5
22

23. ©2023 F5
23
What about Large Language Models?

24. ©2023 F5
24
• Learned language patterns are used to
generate text likely to follow a given
prompt.
• Probabilities of words and phrases are
based on their frequency in the training
data
• LLM’s do not “know” facts and will
hallucinate
• LLM alignment is used to steer the
response to be “appropriate”
Prediction is a core LLM task
What is a Large Language Model (LLM)?

25. ©2023 F5
25
What is open source?
What is open source?
User:
You are a very kind chat
assistant that responds to
user queries.
System:
Open source refers to a
type of software whose
source code is released
under a license…
Assistant:
LLM
Input
Filter
What is open source?
Open source refers to
a type of software …
Output
Filter
LLM Query Flow
Assistant:

26. ©2022 F5
26
LLM Attacks > Prompt Injection
Reference: https://arxiv.org/pdf/2307.15043.pdf
Reference: NVIDIA: Securing LLM Systems Against Prompt Injection

27. ©2022 F5
27
• Privileged Control
• Trust Boundaries
• Dual LLM Pattern
LLM Attacks > Prompt Injection
Countermeasures
Reference: https://arxiv.org/pdf/2307.15043.pdf

28. ©2023 F5
28

29. ©2023 F5
29
Any other security considerations?

30. ©2023 F5
30

31. ©2023 F5
31
Key Takeaways
• Security measures should consider
the entire ML pipeline
• More research + industry
collaboration is needed
• Get involved with communities
working on this

32. ©2023 F5
32
THANK YOU
@abernathyca

33. ©2023 F5
Q & A
33