Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
End to end Machine Learning in
kubernetes with Seldon and
MLFlow
Adaptive. Intelligent.
Agile. Intuitive.
Alive. Inspiring.
About me
Adrian Gonzalez-Martin
Machine Learning Engineer
agm@seldon.io
@kaseyo23
github.com/adriangonz
About Seldon
We are hiring!
seldon.io/careers/
Outline
Example use case.
Training models with MLFlow.
Deploying models with Seldon.
Demo!
Example use case
Wine e-commerce
6.5
2.3
?
https://wine.seldon.io/...
Regression problem where goal is to predict wine quality.
Wine quality dataset
Fixed
Acidity
Volatile
Acidity
Citric
Acid
...
Train two versions of ElasticNet.
Deploy both and do A/B test.
Wine quality dataset
Model A
A/B Test
Router
Model B
ElasticNet
Linear regression with L1 and L2 regularisers.
Two hyperparameters:
Wine quality dataset
Training models with
MLFlow
Keep track of experiments (e.g. versioning).
Optimise hyperparameters at scale.
Share models with the rest of the team.
Pl...
MLtracking
API to track results and hyperparameters.
These can be stored remotely.
MLproject
Define environment, parameters...
MLFlow can help!
MLtracking
Track both hyperparameters:
Track metrics:
MLproject
Pip dependencies and entry point template...
MLproject file and training
MLproject
name: mlflow-talk
conda_env: conda.yaml
entry_points:
main:
parameters:
alpha: float
l1...
MLmodel snapshot
MLmodel
artifact_path: model
flavors:
python_function:
data: model.pkl
env: conda.yaml
loader_module: mlflo...
MLflow UI + MLtrack
Deploying models with
Seldon
Serving infrastructure.
Scaling up (and down!).
Monitoring models.
Updating models.
Deploying models is hard
Analysis Tool...
Inference graph
Seldon/3rd party
component
MLFlow model
API
(REST, gRPC)
Model A
A/B Test
Router
Explainer
Model B
Inferen...
Cloud Native
On-prem
Built on top of Kubernetes Cloud Native APIs.
All major cloud providers are supported.
On-prem provid...
Seldon Architecture
Data scientists,
engineers and
managers
Deployment Controller
(kubectl, CI/CD, Seldon Deploy)
Business...
Inference graph
apiVersion: machinelearning.seldon.io/v1alpha2
kind: SeldonDeployment
metadata:
name: wines-classifier
spec...
Feedback
Kubernetes clusters
running Seldon Core
Inference graphPrediction
Feedback
(reward signal)
Feedback
We can build a rough reward signal using the squared error.
Seldon Analytics
Demo!
https://github.com/adriangonz/mlflow-talk
Thanks!
agm@seldon.io
@kaseyo23
github.com/adriangonz
We are hiring!
seldon.io/careers/
Upcoming SlideShare
Loading in …5
×

of

 Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 1  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 2  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 3  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 4  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 5  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 6  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 7  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 8  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 9  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 10  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 11  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 12  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 13  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 14  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 15  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 16  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 17  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 18  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 19  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 20  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 21  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 22  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 23  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 24  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 25  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 26  Seamless End-to-End Production Machine Learning with Seldon and MLflow Slide 27
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

3 Likes

Share

Download to read offline

Seamless End-to-End Production Machine Learning with Seldon and MLflow

Download to read offline

Deploying and managing machine learning models at scale introduces new complexities. Fortunately, there are tools that simplify this process. In this talk we walk you through an end-to-end hands on example showing how you can go from research to production without much complexity by leveraging the Seldon Core and MLflow frameworks. We will train a set of ML models, and we will showcase a simple way to deploy them to a kubernetes cluster through sophisticated deployment methods, including canary deployments, shadow deployments and we’ll touch upon richer ML graphs such as explainer deployments.

Seamless End-to-End Production Machine Learning with Seldon and MLflow

  1. 1. End to end Machine Learning in kubernetes with Seldon and MLFlow Adaptive. Intelligent. Agile. Intuitive. Alive. Inspiring.
  2. 2. About me Adrian Gonzalez-Martin Machine Learning Engineer agm@seldon.io @kaseyo23 github.com/adriangonz
  3. 3. About Seldon We are hiring! seldon.io/careers/
  4. 4. Outline Example use case. Training models with MLFlow. Deploying models with Seldon. Demo!
  5. 5. Example use case
  6. 6. Wine e-commerce 6.5 2.3 ? https://wine.seldon.io/...
  7. 7. Regression problem where goal is to predict wine quality. Wine quality dataset Fixed Acidity Volatile Acidity Citric Acid ... Sulphates Alcohol Quality 7 0.27 0.36 0.45 8.8 6 6.3 0.3 0.34 0.49 9.5 7 8.1 0.28 0.4 0.44 10.1 1 7.2 0.23 0.32 0.4 9.9 2 7.2 0.23 0.32 0.4 9.9 5 ...
  8. 8. Train two versions of ElasticNet. Deploy both and do A/B test. Wine quality dataset Model A A/B Test Router Model B
  9. 9. ElasticNet Linear regression with L1 and L2 regularisers. Two hyperparameters: Wine quality dataset
  10. 10. Training models with MLFlow
  11. 11. Keep track of experiments (e.g. versioning). Optimise hyperparameters at scale. Share models with the rest of the team. Plain notebooks may not be the best solution!! Training models at scale is hard secret-project ├── eda.ipynb ├── requirements.txt ├── requirements-tf2.txt └── experiments ├── attention-embedd.ipynb ├── attn.ipynb ├── hmm.ipynb ├── LSTM-2.ipynb └── lstm.ipynb
  12. 12. MLtracking API to track results and hyperparameters. These can be stored remotely. MLproject Define environment, parameters and model’s interface. Hyperparameter optimisation can be run Databricks or Kubernetes. MLmodel Snapshot/version of the model. MLFlow can help!
  13. 13. MLFlow can help! MLtracking Track both hyperparameters: Track metrics: MLproject Pip dependencies and entry point template. MLmodel Snapshot of regression coefficients and hyperparameters:
  14. 14. MLproject file and training MLproject name: mlflow-talk conda_env: conda.yaml entry_points: main: parameters: alpha: float l1_ratio: {type: float, default: 0.1} command: "python train.py {alpha} {l1_ratio}" $ mlflow run ./training -P alpha=0.5 $ mlflow run ./training -P alpha=1.0
  15. 15. MLmodel snapshot MLmodel artifact_path: model flavors: python_function: data: model.pkl env: conda.yaml loader_module: mlflow.sklearn python_version: 3.6.9 sklearn: pickled_model: model.pkl serialization_format: cloudpickle sklearn_version: 0.19.1 run_id: 5a6be5a1ef844783a50a6577745dbdc3 utc_time_created: '2019-10-02 14:21:15.783806'
  16. 16. MLflow UI + MLtrack
  17. 17. Deploying models with Seldon
  18. 18. Serving infrastructure. Scaling up (and down!). Monitoring models. Updating models. Deploying models is hard Analysis Tools Serving Infrastructure Monitoring Machine Resource Management Process Management Tools Analysis Tools Serving Infrastructure Monitoring Machine Resource Management Process Management Tools ML Code Data Verification Data Collection Feature Extraction Configuration Seldon can help with these... [1] Hidden Technical Debt in Machine Learning Systems, NIPS 2015
  19. 19. Inference graph Seldon/3rd party component MLFlow model API (REST, gRPC) Model A A/B Test Router Explainer Model B Inference graph (k8s pod)
  20. 20. Cloud Native On-prem Built on top of Kubernetes Cloud Native APIs. All major cloud providers are supported. On-prem providers such as OpenShift are also supported. https://docs.seldon.io/projects/seldon-core/en/latest/examples/notebooks.html
  21. 21. Seldon Architecture Data scientists, engineers and managers Deployment Controller (kubectl, CI/CD, Seldon Deploy) Business Applications REST API or gRPC Kubernetes clusters running Seldon Core Inference Components Kubernetes API Operator Ingress (Ambassador, Istio) Public docker registry Client docker registry Seldon docker registry MLFlow models (S3, GCS, etc.)
  22. 22. Inference graph apiVersion: machinelearning.seldon.io/v1alpha2 kind: SeldonDeployment metadata: name: wines-classifier spec: name: wines-classifier predictors: - graph: children: [] implementation: MLFLOW_SERVER modelUri: gs://seldon-models/mlflow/model-a name: wines-classifier name: model-a replicas: 1 traffic: 50 - graph: children: [] implementation: MLFLOW_SERVER modelUri: gs://seldon-models/mlflow/model-b name: wines-classifier name: model-b replicas: 1 traffic: 50
  23. 23. Feedback Kubernetes clusters running Seldon Core Inference graphPrediction Feedback (reward signal)
  24. 24. Feedback We can build a rough reward signal using the squared error.
  25. 25. Seldon Analytics
  26. 26. Demo! https://github.com/adriangonz/mlflow-talk
  27. 27. Thanks! agm@seldon.io @kaseyo23 github.com/adriangonz We are hiring! seldon.io/careers/
  • NithinBhardwaj

    May. 18, 2020
  • FedericoFalcone5

    Jan. 17, 2020
  • tushar_kale

    Nov. 10, 2019

Deploying and managing machine learning models at scale introduces new complexities. Fortunately, there are tools that simplify this process. In this talk we walk you through an end-to-end hands on example showing how you can go from research to production without much complexity by leveraging the Seldon Core and MLflow frameworks. We will train a set of ML models, and we will showcase a simple way to deploy them to a kubernetes cluster through sophisticated deployment methods, including canary deployments, shadow deployments and we’ll touch upon richer ML graphs such as explainer deployments.

Views

Total views

695

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

47

Shares

0

Comments

0

Likes

3

×