DevSecOps Day 2024 focuses on the integration of Machine Learning Security Operations (MLSecOps) with essential principles such as supply chain vulnerability, model provenance, and governance risk compliance. Key takeaways include the top ten vulnerabilities in ML and the responsible machine learning principles, emphasizing human augmentation, bias evaluation, and explainability. The event also explored strategies for maintaining security throughout the machine learning lifecycle, including monitoring and addressing adversarial risks.

1. DevSecOps Day 2024
A gentle introduction to
MLSecOps

2. DevSecOps Day 2024

3. DevSecOps Day 2024

4. DevSecOps Day 2024

5. DevSecOps Day 2024
Data & Automation Specialist @ Corley Cloud
alessandra.bilardi@corley.it
corley.it
Alessandra Bilardi

6. DevSecOps Day 2024
Data & Automation Specialist @ Corley Cloud
alessandra.bilardi@corley.it
corley.it
Alessandra Bilardi

7. DevSecOps Day 2024
Data & Automation Specialist @ Corley Cloud
alessandra.bilardi@corley.it
corley.it
Alessandra Bilardi

8. DevSecOps Day 2024
Corley Cloud è
Advanced Partner AWS
Soluzioni & servizi offerti da Corley Cloud

9. DevSecOps Day 2024
SUMMARY
From MLOps to MLSecOps
The Responsible Machine Learning Principles
Top 10 Vulnerabilities
Takeaways

10. DevSecOps Day 2024
From MLOps to MLSecOps

11. DevSecOps Day 2024
ML system

12. DevSecOps Day 2024
ML system

13. DevSecOps Day 2024
ML system

14. DevSecOps Day 2024
ML system

15. DevSecOps Day 2024
MLOps

16. DevSecOps Day 2024
➔ CI / CD for software development (code)
➔ Automation of the distribution of the
environment with fault alerts
➔ Versioning of code about each
components
➔ Testing of code
➔ Monitoring instances metrics & logs,
autoscaling
DevOps

17. DevSecOps Day 2024
➔ CI / CD for software development (code)
➔ Automation of the distribution of the
environment with fault alerts
➔ Versioning of code about each
components
➔ Testing of code
➔ Monitoring instances metrics & logs,
autoscaling
➔ CI / CD for software development (code)
➔ Automation of the distribution of the
environment with fault alerts
➔ Versioning of data, model and code
about each machine learning step
➔ Testing of data, model and code
➔ Monitoring instances metrics & logs,
autoscaling, model decay, data bias
DevOps MLOps

18. DevSecOps Day 2024
[S|L] LM system

19. DevSecOps Day 2024
[S|L] LM system

20. DevSecOps Day 2024
[S|L] LM system

21. DevSecOps Day 2024
[S|L] LMOps

22. DevSecOps Day 2024
➔ CI / CD for software development (code)
➔ Automation of the distribution of the
environment with fault alerts
➔ Versioning of code about each
components
➔ Testing of code
➔ Monitoring instances metrics & logs,
autoscaling
➔ CI / CD for software development (code)
➔ Automation of the distribution of the
environment with fault alerts
➔ Versioning of data, model and code
about each machine learning step
➔ Testing of data, model and code
➔ Monitoring instances metrics & logs,
autoscaling, model decay, data bias
DevOps MLOps / [S|L] LMOps

23. DevSecOps Day 2024
MLSecOps

24. DevSecOps Day 2024
Domains
1. Supply Chain
Vulnerability

25. DevSecOps Day 2024
Domains
1. Supply Chain
Vulnerability

26. DevSecOps Day 2024
Domains
1. Supply Chain
Vulnerability

27. DevSecOps Day 2024
Domains ● maintenance
● monitoring
● identify vulnerabilities
● secure software supply chains
● integrity and reliability
● data and infrastructure documented
1. Supply Chain
Vulnerability

28. DevSecOps Day 2024
Domains
1. Supply Chain
Vulnerability
2. Model Provenance

29. DevSecOps Day 2024
Domains
1. Supply Chain
Vulnerability
2. Model Provenance
Pedigree

30. DevSecOps Day 2024
Domains
1. Supply Chain
Vulnerability
2. Model Provenance
3. Governance, Risk &
Compliance (GRC)

31. DevSecOps Day 2024
Domains
1. Supply Chain
Vulnerability
2. Model Provenance
3. Governance, Risk &
Compliance (GRC)
Machine Learning Bill of Materials (MLBoM)

32. DevSecOps Day 2024
With great power comes
great responsibility.
STAN LEE VOLTAIRE

33. DevSecOps Day 2024
Domains
1. Supply Chain
Vulnerability
2. Model Provenance
3. Governance, Risk &
Compliance (GRC)
4. Trusted AI: Bias,
Fairness &
Explainability
https://github.com/Trusted-AI

34. DevSecOps Day 2024
Domains
1. Supply Chain
Vulnerability
2. Model Provenance
3. Governance, Risk &
Compliance (GRC)
4. Trusted AI: Bias,
Fairness &
Explainability
5. Adversarial ML
https://github.com/Trusted-AI

35. DevSecOps Day 2024
Domains
1. Supply Chain
Vulnerability
2. Model Provenance
3. Governance, Risk &
Compliance (GRC)
4. Trusted AI: Bias,
Fairness &
Explainability
5. Adversarial ML
https://adversarial-robustness-toolbox.readthedocs.io/

36. DevSecOps Day 2024
Machine learning Security Operations
➔ Supply Chain Vulnerability with
maintenance and monitoring for
software development (code)
➔ Model Provenance lineage of model
➔ Governance, Risk & Compliance with
Machine Learning Bill of Materials
(MLBoM) for data, model, algos & code
➔ Trusted AI with Fairness &
Explainability tools for data & model
➔ Adversarial Machine Learning as
another tool against malicious attacks

37. DevSecOps Day 2024
MLSecOps DevSecOps
➔ Supply Chain Vulnerability with
maintenance, monitoring and
application security testing (DAST, SAST,
IAST, SCA) for software development
(code)
➔ Governance, Risk & Compliance for
code and infrastructure
➔ WAF to manage filtering, injection,
intrusion prevention, inspections, ..
➔ Mesh to manage traffic routing
➔ DDos mitigation
➔ Supply Chain Vulnerability with
maintenance and monitoring for
software development (code)
➔ Model Provenance lineage of model
➔ Governance, Risk & Compliance with
Machine Learning Bill of Materials
(MLBoM) for data, model, algos & code
➔ Trusted AI with Fairness &
Explainability tools for data & model
➔ Adversarial Machine Learning as
another tool against model attacks

38. DevSecOps Day 2024
AI system

39. DevSecOps Day 2024
The Responsible ML Principles

40. DevSecOps Day 2024
Principles
I commit ..
1. Human augmentation

41. DevSecOps Day 2024
Principles
I commit ..
1. Human augmentation
2. Bias evaluation

42. DevSecOps Day 2024
Principles
I commit ..
1. Human augmentation
2. Bias evaluation
3. Explainability by
justification

43. DevSecOps Day 2024
Principles
I commit ..
1. Human augmentation
2. Bias evaluation
3. Explainability by
justification
4. Reproducible
operations

44. DevSecOps Day 2024
Principles
I commit ..
1. Human augmentation
2. Bias evaluation
3. Explainability by
justification
4. Reproducible
operations
5. Displacement strategy

45. DevSecOps Day 2024
Principles
I commit ..
1. Human augmentation
2. Bias evaluation
3. Explainability by
justification
4. Reproducible
operations
5. Displacement strategy
6. Practical accuracy

46. DevSecOps Day 2024
Principles
I commit ..
1. Human augmentation
2. Bias evaluation
3. Explainability by
justification
4. Reproducible
operations
5. Displacement strategy
6. Practical accuracy
7. Trust by privacy

47. DevSecOps Day 2024
Principles
I commit ..
1. Human augmentation
2. Bias evaluation
3. Explainability by
justification
4. Reproducible
operations
5. Displacement strategy
6. Practical accuracy
7. Trust by privacy
8. Data risk awareness

48. DevSecOps Day 2024
Top 10 Vulnerabilities

49. DevSecOps Day 2024
OWASP Vulnerability
1. Broken Access Control
2. Cryptographic Failures
3. Injection
4. Insecure Design
5. Security Misconfigurations
6. Vulnerable and Outdated
Components
7. Identification and Auth Failures
8. Software and Data Integrity
Failures
9. Logging and Monitoring Failures
10. Server-side Request Forgery

50. DevSecOps Day 2024
OWASP Vulnerability - MLSecOps Equivalent
1. Unrestricted Model Endpoints
2. Access to Model Artifacts
3. Artifact Exploit Injection
4. Insecure ML Systems/Pipeline Design
5. Data and ML Infra Misconfigurations
6. Supply Chain Vulnerabilities
in ML Code
7. IAM and RBAC Failures for ML Services
8. ML infra / ETL / CI / CD integrity
Failures
9. Observability, Reproducibility & Lineage
10. ML-Server Side Request Forgery
1. Broken Access Control
2. Cryptographic Failures
3. Injection
4. Insecure Design
5. Security Misconfigurations
6. Vulnerable and Outdated
Components
7. Identification and Auth Failures
8. Software and Data Integrity
Failures
9. Logging and Monitoring Failures
10. Server-side Request Forgery

51. DevSecOps Day 2024
Takeaways

52. DevSecOps Day 2024
Concepts
MLSecOps Domains
1. Supply Chain Vulnerability
2. Model Provenance
3. Governance, Risk & Compliance
4. Trusted AI
5. Adversarial Machine Learning
The Responsible ML Principles
1. Human augmentation
2. Bias evaluation
3. Explainability by justification
4. Reproducible operations
5. Displacement strategy
6. Practical accuracy
7. Trust by privacy
8. Data risk awareness

53. DevSecOps Day 2024
Thanks
for listening!

54. DevSecOps Day 2024
References
● https://mlsecops.com/
● https://huntr.com/
● https://owasp.org/www-project-top-ten/
● https://ethical.institute/
● https://github.com/EthicalML/fml-security