Because our models and pipelines don’t usually run in production, it's natural to put less scrutiny into the security of the systems and the code. However, vulnerabilities in our data architecture, software architecture, or network design can expose critical company IP or personal data to hackers or fraudsters. Vulnerabilities in the open-source packages we use to build our models can be exploited as well. This talk covers considerations around security that should be central to anyone building ML/AI solutions.

1. 1 © 2022 Anaconda
What Vulnerabilities?
How (And Why) to Secure Your ML/AI Solutions
Kevin Goldsmith, CTO
October 7, 2022

2. 2 © 2022 Anaconda
We all (hopefully) understand that security is
important for our production systems

3. 3 © 2022 Anaconda
While hardening our production data and systems,
we don’t often think about our ML and AI pipelines
because they are not directly consumed by users.
EDW
Backoffice
Systems
Data Scientists
Data Models
Production Customers

4. 4 © 2022 Anaconda
While hardening our production data and systems,
we often don’t think about our ML and AI pipelines
because they are not directly consumed by users.
EDW
Backoffice
Systems
Data Scientists
Data Models
Production Customers

5. 5 © 2022 Anaconda
While hardening our production data and systems,
we often don’t think about our ML and AI pipelines
because they are not directly consumed by users.
EDW
Backoffice
Systems
Data Scientists
Data Models
Production Customers

6. 6 © 2022 Anaconda
While hardening our production data and systems,
we often don’t think about our ML and AI pipelines
because they are not directly consumed by users.
EDW
Backoffice
Systems
Data Scientists
Data Models
Production Customers
EVASION ATTACKS
POISONING ATTACKS
INFERENCE ATTACKS
TROJAN ATTACKS
SUPPLY CHAIN ATTACKS

7. 7 © 2022 Anaconda
Protecting your systems

8. 8 © 2022 Anaconda
Evasion/Adversarial Attacks
Protecting your systems
Designing an input to trick ML classification

9. 9 © 2022 Anaconda
Evasion/Adversarial Attacks
Protecting your systems
● Formal methods
○ Good for mission-critical systems where attacks can mean significant costs or life-threatening
outcomes
○ Expensive and time-consuming
● Empirical defenses
○ Adversarial training
○ Input Modification (aka: input cleansing)
○ Drift/generalized detection

10. 10 © 2022 Anaconda
Protecting your systems
Membership Inference Attacks
Devine the input data by observing the output of the model against a set of inputs.
Training
data

11. 11 © 2022 Anaconda
Protecting your systems
Membership Inference Attacks
● Adversarial Training
● Avoid overfitting
○ Use Regularization Techniques

12. 12 © 2022 Anaconda
Protecting your systems
Poisoning Attacks
Injecting bad training data to change model predictions/classifications
Training
data

13. 13 © 2022 Anaconda
Protecting your systems
Poisoning Attacks
● Establish data provenance
○ Manual curation / known-good training data sets
● Robustify models
○ Iteratively minimize trimmed loss and retrain on random sets of clean and poisoned data

14. 14 © 2022 Anaconda
Protecting your systems
Trojan Attacks
Embedding malicious weights into a Neural Network to create a pre-defined output
for certain inputs and expected outputs otherwise.

15. 15 © 2022 Anaconda
Protecting your systems
Trojan Attacks
● Establish training data provenance
○ Create policies around using publicly available data sets
● Establish model provenance
○ Discourage use of pre-trained models for critical applications

16. 16 © 2022 Anaconda
Supply Chain Attacks

17. 17 © 2022 Anaconda
79%
OSS provides flexibility to customize
solutions to meet company’s needs
77%
OSS ensures my
organization has access to
the latest innovations
76%
OSS simplifies the process of
adopting a hybrid cloud
infrastructure
Advantages of Open-Source Software
92%
of all applications contain open-source
software
97%
of codebases contain open-
source software
95%
of IT organizations rely on
open-source software
From a RedHat survey of IT professionals:
From leading OSS security research:
Sources: RedHat, Linux Foundation, Synopsys, Snyk, Sonatype

18. 18 © 2022 Anaconda
650%
Increase in cyberattacks targeting
open-source
$1.4MM
Average ransomware remediation
costs
49%
of organizations have a security
policy addressing open-source
Risks and Challenges
73%
of organizations are searching for
best practices to improve their
open-source security
Sources: Linux Foundation, NVD, Snyk, Sonatype, Sophos

19. 19 © 2022 Anaconda
>395,000
Projects
46%
With Vulnerabilities

20. 20 © 2022 Anaconda
46%
With Vulnerabilities
>395,000
Projects
11%
With Critical Vulnerabilities

21. 21 © 2022 Anaconda
It’s never just the packages you are installing

22. 22 © 2022 Anaconda
15,980
Unique Data Science Packages
Downloaded in August
1949
Had Known Vulnerabilities
556
Of the vulnerabilities were scored as
critical
We looked at the top 10 Canadian Companies
by Market Capitalization
4-202
Number of packages downloaded
per company with critical
vulnerabilities

23. 23 © 2022 Anaconda
Avoiding Supply Chain Attacks
● Know what you are using
○ Inventory not only the root packages, but also all their dependencies
● Monitor new Common Vulnerabilities and Exposures (CVEs)
○ Identify new vulnerabilities in existing packages
● Use trusted repositories
○ Avoid typosquating and account hijack attacks
● Use trusted builds of packages
○ Build yourself or use a trusted build partner that signs their artifacts

24. 24 © 2022 Anaconda
CVE growth
25,000
20,000
15,000
10,000
5,000
0
2016 2017 2018 2019 2020 2021
75% FALSE POSITIVES

25. 25 © 2022 Anaconda
Anaconda - CVE Curation Flow
The National Institute of
Standards and
Technology(NIST) National
Vulnerability Database
(NVD)
CVE Data Source
Associating NVD CVE data
with packages in the
Anaconda Repository
Automated Matching
Anaconda staff review
NVD CVE data for accuracy
and then categorize, refine
and improve the reported
information. If possible,
CVEs are patched
Human Curation
Accurate CVE metadata
allows organizations to
filter out OSS packages
that don’t meet their
security requirements
Refined CVE Metadata
Goal: high-quality, accurate, and dependable CVE information
Performed by Anaconda Distribution Team

26. 26 © 2022 Anaconda
Elements of OSS Security and Governance
Are you able to proactively keep high-risk
components out of your pipeline?
Do you have a trusted repository?
Do you know what is in your open-source pipeline?
Are you able to identify high-risk components in
your pipeline?

27. 27 © 2022 Anaconda
Bringing open-source innovation
securely to the enterprise
✓ Centralized, secure, and trusted source for OSS
✓ Visibility into the software components of your pipeline
✓ Access to Anaconda-curated vulnerability data
✓ Automate enforcement of security and licensing requirements
✓ Create and maintain an audit trail
Act on high-risk components with access
to curated vulnerability data and details
Block high-risk components with
customizable and automated
security policy enforcement

28. Questions?

29. Thank You!
Kevin Goldsmith
Chief Technology Officer

30. About Anaconda
With more than 30 million users, Anaconda is the world’s most
popular data science platform and the foundation of modern
machine learning. We pioneered the use of Python for data
science, champion its vibrant community, and continue to
steward open-source projects that make tomorrow’s
innovations possible. Our enterprise-grade solutions enable
corporate, research, and academic institutions around the world
to harness the power of open-source for competitive
advantage, groundbreaking research, and a better world.
Visit https://www.anaconda.com to learn more.