Bodywork - GitOps for Machine Learning
•Download as PPTX, PDF•
0 likes•379 views
The document discusses continuous deployment for machine learning projects using Bodywork. Bodywork enables CI/CD for ML projects by allowing teams to define workflows in a Git repository that can deploy models, pipelines, and services to Kubernetes. It tackles common deployment challenges for ML like managing container images and infrastructure, and allows teams to focus on their code. The talk includes demos of sample ML projects deployed with Bodywork, like serving a model with a REST API and a train-and-serve pipeline.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Bodywork - GitOps for Machine Learning
Similar to Bodywork - GitOps for Machine Learning (20)
Recently uploaded
Recently uploaded (20)
Bodywork - GitOps for Machine Learning
- 1. Bodywork – GitOps for Machine Learning Continuous deployment for data science and machine learning teams Alex Ioannides, March 2021
- 2. 1.Continuous deployment in machine learning 2.Why machine learning projects are hard to deploy 3.What’s Bodywork and how does it help? 4.Arriving at GitOps 5.Case studies and demos AGENDA
- 3. • Co-founder of Bodywork Machine Learning, the creators of Bodywork. • ML engineer for Oracle AI Apps division, focusing on MLOps. • Built the ML functions for Perfect Channel and LiveMore Capital. • 8 years in financial services – Credit Suisse, Standard Bank, Moodys. • PhD in computational neuroscience from UCL. • Recovering theoretical physicist (going back a long time now…). ABOUT ME
- 4. I am not a software engineer, and this is not going to be a lecture on DevOps best-practices for data scientists and ML engineers. I have, however, worked with some brilliant developers and learnt a lot from them in the process. This talk will be centered from a ML engineer’s frame-of-reference – i.e., what are useful best practices for ML and why. ABOUT ME
- 5. CI / CD CONTINUOUS DEPLOYMENT IN ML
- 6. CONTINUOUS DEPLOYMENT IN ML Continuous Integration
- 7. CONTINUOUS DEPLOYMENT IN ML Aside - Writing tests for ML Systems Example Unit Tests for ML: • Feature transformation pipelines yield expected results. • Training routines yield ‘valid’ models for ‘valid’ data. • Unseen categories and/or outliers are handled gracefully. Example Integration Tests for ML: • You can read-from and write-to object storge or your model registry, feature store, etc. • Requests to scoring services with REST APIs yield expected responses. ➡️ “Effective Testing for Machine Learning Systems” by Jeremy Jordan
- 8. CONTINUOUS DEPLOYMENT IN ML Continuous Deployment ?
- 9. CONTINUOUS DEPLOYMENT IN ML Continuous Deployment – push models to cloud object storage
- 10. CONTINUOUS DEPLOYMENT IN ML Continuous Deployment – push models to a registry
- 11. CONTINUOUS DEPLOYMENT IN ML Continuous Deployment – push container images to a cloud platform
- 12. CONTINUOUS DEPLOYMENT IN ML Continuous Deployment – push container images to a Kubernetes cluster
- 13. CONTINUOUS DEPLOYMENT IN ML Aside – Docker and Kubernetes are ideal for building ML Systems Docker: • Reproducible environments. • Compose ML pipelines using containers as building-blocks. Kubernetes: • Provides all the resources you could want for building a MLOps platform – e.g., jobs, services and easy networking. • Resilience and horizontal-scaling are built-in from the bottom-up. ➡️ “Deploying Python ML Models with Flask, Docker and Kubernetes" by Me
- 14. Example – serve a pre-trained model via a microservice with a REST API WHY ML PROJECTS ARE HARD TO DEPLOY
- 15. WHY ML PROJECTS ARE HARD TO DEPLOY Example – serve a pre-trained model via a microservice with a REST API • Multiple points of failure. • Requires a more-than-basic understanding of Docker and Linux. • Needs experience with container orchestration (e.g., Kubernetes). • Doesn’t scale easily. • Maintaining Docker images is an extra responsibility for ML engineers. • This is not Machine Learning - this is DevOps.
- 16. WHAT’S BODYWORK AND HOW DOES IT HELP?
- 17. WHAT’S BODYWORK AND HOW DOES IT HELP? Continuous Deployment – from GitHub to Kubernetes with Bodywork
- 18. WHAT’S BODYWORK AND HOW DOES IT HELP? Tackle deployment problems head-on and separately from project codebase: • Create a generic Linux container with Python and Git installed. • Use Git to pull project code into the cluster environment and then dynamically install requirements - removes the need to build, push and manage container images on a project-by-project basis. • Each stage runs a Python executable defining a task – either as a discrete batch job or the deployment of a long-running service. • Combine multiple stages into workflows, using a workflow-controller.
- 19. Manage complex pipelines and service topologies, with high concurrency. WHAT’S BODYWORK AND HOW DOES IT HELP?
- 20. Use existing code with Bodywork’s project format – no new APIs to learn! WHAT’S BODYWORK AND HOW DOES IT HELP?
- 21. WHAT’S BODYWORK AND HOW DOES IT HELP?
- 22. Enables CI/CD for ML projects. WHAT’S BODYWORK AND HOW DOES IT HELP?
- 23. ARRIVING AT GITOPS ➡️ www.gitops.tech
- 24. ARRIVING AT GITOPS “The core idea of GitOps is having a Git repository that always contains declarative descriptions of the infrastructure currently desired in the production environment and an automated process to make the production environment match the described state in the repository. If you want to deploy a new application or update an existing one, you only need to update the repository - the automated process handles everything else. It’s like having cruise control for managing your applications in production.”
- 25. (1) Serving a model via a microservice with a REST API CASE STUDIES AND DEMOS ➡️ https://github.com/bodywork-ml/bodywork-serve-model-project
- 26. CASE STUDIES AND DEMOS
- 27. (1) Serving a model via a microservice with a REST API CASE STUDIES AND DEMOS ➡️ https://github.com/bodywork-ml/bodywork-serve-model-project
- 28. (1) Serving a model via a microservice with a REST API CASE STUDIES AND DEMOS ➡️ https://github.com/bodywork-ml/bodywork-serve-model-project
- 29. (2) Train-and-serve ML pipeline CASE STUDIES AND DEMOS ➡️ https://github.com/bodywork-ml/bodywork-ml-pipeline-project
- 30. (2) Train-and-serve ML pipeline CASE STUDIES AND DEMOS ➡️ https://github.com/bodywork-ml/bodywork-ml-pipeline-project
- 31. CASE STUDIES AND DEMOS
- 32. (2) Train-and-serve ML pipeline CASE STUDIES AND DEMOS ➡️ https://github.com/bodywork-ml/bodywork-ml-pipeline-project
- 33. (3) Jupyter pipelines CASE STUDIES AND DEMOS ➡️ https://github.com/bodywork-ml/bodywork-jupyter-pipeline-project
- 34. (4) ML Dashboards with Plotly CASE STUDIES AND DEMOS ➡️ https://github.com/bodywork-ml/bodywork-ml-dashboard-project
- 35. (4) ML Dashboards with Plotly CASE STUDIES AND DEMOS ➡️ https://github.com/bodywork-ml/bodywork-ml-dashboard-project
Editor's Notes
- ”Thank you for inviting me to speak to today.”
- “I’m going to talk about deploying ML projects to ‘production environments” We’ll focus on how this process can be automated, so we can deploy continuously. We’ll look at the sorts of problems that are often encountered. Then I’ll introduce Bodywork – an open-source framework for continuously deploying ML projects to Kubernetes. I’ll cover how it works and in particular how it uses a pattern called GitOps.
- “A little bit about me before we get started” Currently trying to see how far we can take Bodywork and the ideas within it. Oracle was where I first encountered ML deployment at scale and had to confront the problems that Bodywork solves.
- “Before we get started, I was to set expectations about where we’re going to go.”
- “So, what do we mean when we talk about continuous deployment?” - Most likely to have heard about ‘CICD pipelines’, in which case Continuous Deployment is the CD in CICD. - Let’s start by looking at CICD and what it means in the context of ML projects.
- “Here is CI workflow that I’ve used many times in the past.” Describe what is happening here. Continuous integration is the ability to continuously integrate new code into a project. Closely liked to agile development practices.
- “Before we move to CD, It’s worth spending a couple of minutes talking about tests for ML systems.” - Based on my experinces.
- “We’ve now arrived at CD. We’ve integrated new code – now what?” - Lets talk about some options…
- - e.g. your team just want to publish models for down-stream users to pick-up and use as they wish.
- - e.g. same as before, but you want to work with a model registry to provide user with an interface built for ML models.
- - e.g. you want to deploy containerised ML applications, with some guarantees on reproducibility, so you also need a container orchestration platform.
- - e.g. you want to deploy containerised ML applications, with some guarantees on reproducibility, so you also need a container orchestration platform that is agnostic to cloud provider.
- “I love Kuberentes and I want to spend a minute telling you why.” - But, I do not think that DS and MLEs need to learn Kubernetes and this has been one of the driving forces behind the development of Bodywork.
- “Let’s drill-down into what a deployment pipeline for a simple ML system could look like.” - Describe the use-case and how I’d get it into production.
- “Based on my experiences…” - And this is just for a simple use-case.
- “We built Bodywork to tackle these ML deployment problems head-on.” Solves the Docker + Kubernetes knowledge-gap. Removes the brittle build-push-deploy pipeline that can be a burden on DS and MLEs. Let DS and MLEs focus on what they do best - ML!
- “Bodywork replaces the CD part of the CICD pipeline and takes full responsibility for deploying your ML system.” - Eliminates all build-and-push responsibilities.
- “Bodywork solves ML deployment problems, by using the following patterns.” - Why do we have workflows so we can build pipelines
- “Bodywork can handle complex deployments graphs, so it can grow with your projects.” - We built a lightweight workflow-controller to avoid having to deploy Apache Airfow or Argo Workflows to manage deployment DAGs, so that we could make Bodywork as simple as possible for the vast majority of use-cases. - From batch-scoring to training models in parallel and deploying complex service topologies – e.g. A/B testing of new models (services).
- “We wanted to minimise any interference that using Bodywork, would have on users.” Wanted to make existing code re-usable. Did not want people to have to learn any new APIs and engineer their code around them. The only requirements are that your project code has to available on a remote Git repo and that each deployment stage has to be within its own directory, with some lightweight config.
- “Here’s what a train-and-serve pipeline looks like, using Bodywork.”
- “Now, let’s go back to where we started and think about what CICD looks like with Bodywork in-the-loop.” The CI part is now the last part of the pipeline and everything to do with your ML system is contained within your Git repo. Periodically re-deploys your project, using the latest state of the codebase.
- “And so we have now arrived at GitOps.” This was not intentional, but I realised it after I started reading this book. EVERYTHING you need to describe your entire ML system is contained in the Git repo. Two deployment ‘modes’ push or pull and Bodywork supports both (deployment or cronjob)
- “We took this one step further to include the ML application itself.”
- “Bodywork is distributed as a Python package that exposes a command-line interface for configuring Kubernetes to deploy your projects.”