Download as PDF, PPTX
![Serving infrastructure.
Scaling up (and down!).
Monitoring models.
Updating models.
Deploying models is hard
Analysis Tools
Serving
Infrastructure
Monitoring
Machine
Resource
Management
Process
Management
Tools
Analysis Tools
Serving
Infrastructure
Monitoring
Machine
Resource
Management
Process
Management Tools
ML
Code
Data Verification
Data Collection
Feature Extraction
Configuration
Seldon can help with these...
[1] Hidden Technical Debt in Machine Learning Systems, NIPS 2015](https://image.slidesharecdn.com/adriangonzalezmartin-191029235514/75/Seamless-End-to-End-Production-Machine-Learning-with-Seldon-and-MLflow-18-2048.jpg)
![Inference graph
apiVersion: machinelearning.seldon.io/v1alpha2
kind: SeldonDeployment
metadata:
name: wines-classifier
spec:
name: wines-classifier
predictors:
- graph:
children: []
implementation: MLFLOW_SERVER
modelUri: gs://seldon-models/mlflow/model-a
name: wines-classifier
name: model-a
replicas: 1
traffic: 50
- graph:
children: []
implementation: MLFLOW_SERVER
modelUri: gs://seldon-models/mlflow/model-b
name: wines-classifier
name: model-b
replicas: 1
traffic: 50](https://image.slidesharecdn.com/adriangonzalezmartin-191029235514/75/Seamless-End-to-End-Production-Machine-Learning-with-Seldon-and-MLflow-22-2048.jpg)
Uploaded byDatabricks
Seamless End-to-End Production Machine Learning with Seldon and MLflow
This document discusses using MLFlow to train machine learning models and Seldon to deploy them in a Kubernetes environment. It provides an example of using a wine quality dataset to train two ElasticNet regression models with MLFlow and deploy them for an A/B test using Seldon. Key steps covered include tracking experiments and hyperparameters with MLFlow, defining the model interface with an MLproject file, and creating the inference graph in Seldon to route traffic between the two models and provide a feedback loop.
More Related Content
Similar to Seamless End-to-End Production Machine Learning with Seldon and MLflow
Seamless End-to-End Production Machine Learning with Seldon and MLflow
- 1. End to endMachine Learning in kubernetes with Seldon and MLFlow Adaptive. Intelligent. Agile. Intuitive. Alive. Inspiring.
- 2. About me Adrian Gonzalez-Martin MachineLearning Engineer agm@seldon.io @kaseyo23 github.com/adriangonz
- 3. About Seldon We arehiring! seldon.io/careers/
- 4. Outline Example use case. Trainingmodels with MLFlow. Deploying models with Seldon. Demo!
- 5.
- 6.
- 7. Regression problem wheregoal is to predict wine quality. Wine quality dataset Fixed Acidity Volatile Acidity Citric Acid ... Sulphates Alcohol Quality 7 0.27 0.36 0.45 8.8 6 6.3 0.3 0.34 0.49 9.5 7 8.1 0.28 0.4 0.44 10.1 1 7.2 0.23 0.32 0.4 9.9 2 7.2 0.23 0.32 0.4 9.9 5 ...
- 8. Train two versionsof ElasticNet. Deploy both and do A/B test. Wine quality dataset Model A A/B Test Router Model B
- 9. ElasticNet Linear regression withL1 and L2 regularisers. Two hyperparameters: Wine quality dataset
- 10.
- 11. Keep track ofexperiments (e.g. versioning). Optimise hyperparameters at scale. Share models with the rest of the team. Plain notebooks may not be the best solution!! Training models at scale is hard secret-project ├── eda.ipynb ├── requirements.txt ├── requirements-tf2.txt └── experiments ├── attention-embedd.ipynb ├── attn.ipynb ├── hmm.ipynb ├── LSTM-2.ipynb └── lstm.ipynb
- 12. MLtracking API to trackresults and hyperparameters. These can be stored remotely. MLproject Define environment, parameters and model’s interface. Hyperparameter optimisation can be run Databricks or Kubernetes. MLmodel Snapshot/version of the model. MLFlow can help!
- 13. MLFlow can help! MLtracking Trackboth hyperparameters: Track metrics: MLproject Pip dependencies and entry point template. MLmodel Snapshot of regression coefficients and hyperparameters:
- 14. MLproject file andtraining MLproject name: mlflow-talk conda_env: conda.yaml entry_points: main: parameters: alpha: float l1_ratio: {type: float, default: 0.1} command: "python train.py {alpha} {l1_ratio}" $ mlflow run ./training -P alpha=0.5 $ mlflow run ./training -P alpha=1.0
- 15. MLmodel snapshot MLmodel artifact_path: model flavors: python_function: data:model.pkl env: conda.yaml loader_module: mlflow.sklearn python_version: 3.6.9 sklearn: pickled_model: model.pkl serialization_format: cloudpickle sklearn_version: 0.19.1 run_id: 5a6be5a1ef844783a50a6577745dbdc3 utc_time_created: '2019-10-02 14:21:15.783806'
- 16. MLflow UI +MLtrack
- 17.
- 18. Serving infrastructure. Scaling up(and down!). Monitoring models. Updating models. Deploying models is hard Analysis Tools Serving Infrastructure Monitoring Machine Resource Management Process Management Tools Analysis Tools Serving Infrastructure Monitoring Machine Resource Management Process Management Tools ML Code Data Verification Data Collection Feature Extraction Configuration Seldon can help with these... [1] Hidden Technical Debt in Machine Learning Systems, NIPS 2015
- 19. Inference graph Seldon/3rd party component MLFlowmodel API (REST, gRPC) Model A A/B Test Router Explainer Model B Inference graph (k8s pod)
- 20. Cloud Native On-prem Built ontop of Kubernetes Cloud Native APIs. All major cloud providers are supported. On-prem providers such as OpenShift are also supported. https://docs.seldon.io/projects/seldon-core/en/latest/examples/notebooks.html
- 21. Seldon Architecture Data scientists, engineersand managers Deployment Controller (kubectl, CI/CD, Seldon Deploy) Business Applications REST API or gRPC Kubernetes clusters running Seldon Core Inference Components Kubernetes API Operator Ingress (Ambassador, Istio) Public docker registry Client docker registry Seldon docker registry MLFlow models (S3, GCS, etc.)
- 22. Inference graph apiVersion: machinelearning.seldon.io/v1alpha2 kind:SeldonDeployment metadata: name: wines-classifier spec: name: wines-classifier predictors: - graph: children: [] implementation: MLFLOW_SERVER modelUri: gs://seldon-models/mlflow/model-a name: wines-classifier name: model-a replicas: 1 traffic: 50 - graph: children: [] implementation: MLFLOW_SERVER modelUri: gs://seldon-models/mlflow/model-b name: wines-classifier name: model-b replicas: 1 traffic: 50
- 23. Feedback Kubernetes clusters running SeldonCore Inference graphPrediction Feedback (reward signal)
- 24. Feedback We can builda rough reward signal using the squared error.
- 25.
- 26.
- 27.