Dmitry Spodarets: Modern MLOps toolchain 2023 AI & BigData Online Day 2023 Spring Website - www.aiconf.com.ua Youtube - https://www.youtube.com/startuplviv FB - https://www.facebook.com/aiconf

1. Modern MLOps toolchain
2023

2. About Me
Dmytro Spodarets
● DevOps Architect at Grid Dynamics
● Founder and chief editor of Data Phoenix
AWS | Infrastructure | DevOps/MLOps

4. Agenda
● What is MLOps?
● DevOps vs MLOps
● MLOps Stack
● Use cases:
○ From research to production
○ Versioning & retraining
○ IaaC & K8s
○ Using GitOps for Machine Learning

6. CRISP-DM

7. Hidden Technical Debt in Machine Learning Systems
https://papers.nips.cc/paper/2015/file/86df7dcfd896fcaf2674f757a2463eba-Paper.pdf

11. The goal of MLOps is to reduce technical friction to get the model from an idea
into production in the shortest possible time with as little risk as possible.

12. DevOps vs MLOps

13. MLOps is not a single tool or platform

14. MLOps is about agreeing to do ML the right way and then supporting it.

15. A few shared principles will take you a long way…
ML should be collaborative ML should be reproducible ML should be continuous ML should be tested &
monitored

16. A few shared principles will take you a long way…
ML should be collaborative ML should be reproducible ML should be continuous ML should be tested &
monitored
Shared Infrastructure
Versioning for Code,
Data and Metadata
Machine Learning Pipelines Model Deployment
and Monitoring

17. Continuous X
MLOps is an ML engineering culture that includes the following practices:
● Continuous Integration (CI) extends the testing and validating code and
components by adding testing and validating data and models.
● Continuous Delivery (CD) concerns with delivery of an ML training pipeline
that automatically deploys another the ML model prediction service.
● Continuous Training (CT) is unique to ML systems property, which
automatically retrains ML models for re-deployment.
● Continuous Monitoring (CM) concerns with monitoring production data and
model performance metrics, which are bound to business metrics.

19. MLOps levels
● Level 0: No MLOps
● Level 1: DevOps no MLOps
● Level 2: Automated Training
● Level 3: Automated Model
Deployment
● Level 4: Full MLOps Automated
Retraining
https://learn.microsoft.com/en-us/azure/architecture/example-scenario/mlops/mlops-maturity-model

20. MLOps Stack

23. AWS ML Stack

24. Use cases
○ From research to production
○ Versioning & retraining
○ IaaC & K8s
○ Using GitOps for Machine Learning

25. From research to production

26. From research to production

27. Data versioning & retraining

28. Data versioning & retraining

29. IaaC & K8s

30. Using GitOps for Machine Learning
https://docs.dstack.ai/

32. Large language models / Generative models

33. 3D-parallelism
https://aws.amazon.com/blogs/machine-learning/training-large-language-models-on-amazon-sagemaker-best-practices/

34. Scalable HPC

35. Questions?
Dmytro Spodarets
d.spodarets@dataphoenix.info
https://dataphoenix.info

36. https://www.eventbrite.com/o/data-phoenix-events-23453295848
https://dataphoenix.info/subscribe/