Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Productionzing ML Model using
MLflow Model Serving
• Nagaraj Sengodan
• Nitin Raj Soundararajan
S Nagaraj has helped build the larger
enterprise-wide distributed systems.
Senior Manager in Architecture and
Engineering ...
Agenda
§ MLflow
§ MLflow Serving
§ Manage Served Versions
§ Monitor Served Models
§ Customize Serving Cluster
§ Q & A
MLFlow
• Open machine learning platform
• Works with any ML Library & Language
• Runs the same way anywhere (e.g. any clou...
MLFlow
Machine Learning
lifecycle
MLflow Tracking - Record and query experiments: code, data, config, and
results
MLflow P...
MLFlow
AutoML
End-to-End ML Lifecycle
ML Runtime and
Environments
Batch and
Streaming
Online Serving
Data Science Workspac...
MLFlow – Model Serving
Prep Data Build Model Deploy/Monitor Model
AutoML
End-to-End ML Lifecycle
Batch and
Streaming
Onlin...
Serving
MLflow Serving
• Expose Mlflow model predictions as REST endpoint
• Small cluster is automatically provisioned
• HTTP endp...
Model Serving from Model Registry
Models
Flavor 2
Flavor 1
Custom
Models
Model Serving from Model Registry
Models Tracking
Flavor 2
Flavor 1
Custom
Models
Parameters Metrics Artifacts
Models
Meta...
Model Serving from Model Registry
Staging Production Archived
Data Scientists Deployment Engineers
v1
v2
v3
Models Trackin...
Model serving from Model Registry
Staging Production Archived
Data Scientists Deployment Engineers
v1
v2
v3
Models Trackin...
MLFlow Serving
Demo
Manage Served Versions
• All active (non-archived) model versions are deployed
• Manage model access rights
• Source deplo...
Monitor Served Models
• Displays status indicators for the serving cluster as well as
individual model versions
• Inspect ...
Customize Serving Cluster
• Modify the memory size and number of cores of a serving
cluster
• Ability to add the tags
• Ab...
Demo Code & Deck
• https://github.com/KRSNagaraj/Mod
elServing-DataAISummit2021
Q & A
Feedback
Your feedback is important to us.
Don’t forget to rate and review the sessions.
You’ve finished this document.
Download and read it offline.
Upcoming SlideShare
What to Upload to SlideShare
Next
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

Share

Productionzing ML Model Using MLflow Model Serving

Download to read offline

Productionzing ML Models are needs to ensure model integrity while it efficiently replicate runtime environments across servers besides it keep track of how each of our models were created. It helps us better trace the root cause of changes and issues over time as we acquire new data and update our model. We have greater accountability over our models and the results they generate.

MLflow Model Serving delivers cost-effective and on-click deployment of model for real-time inferences. Also the Model Version deployed in the Model Serving can also be conveniently managed with MLflow Model Registry. We will going to cover following topics Deployment, Consumption and Monitoring. For deployment, we will demo the different version deployment and validate the deployment. For consumption, we demo connecting power bi and generate prediction report using ML Model deployed in MLflow serving. Lastly will wrap up with managing the MLflow serving like, access rights and monitoring capabilities.

  • Be the first to like this

Productionzing ML Model Using MLflow Model Serving

  1. 1. Productionzing ML Model using MLflow Model Serving • Nagaraj Sengodan • Nitin Raj Soundararajan
  2. 2. S Nagaraj has helped build the larger enterprise-wide distributed systems. Senior Manager in Architecture and Engineering team, helping envision and deliver the future for enterprise analytics via Databricks and Azure Synapse. Nitin Raj Soundararajan is a technical consultant focusing on advanced data analytics, data engineering, cloud scale analytics and data science.
  3. 3. Agenda § MLflow § MLflow Serving § Manage Served Versions § Monitor Served Models § Customize Serving Cluster § Q & A
  4. 4. MLFlow • Open machine learning platform • Works with any ML Library & Language • Runs the same way anywhere (e.g. any cloud) • Open interface design (use with any code you already have)
  5. 5. MLFlow Machine Learning lifecycle MLflow Tracking - Record and query experiments: code, data, config, and results MLflow Projects - Package data science code in a format to reproduce runs on any platform MLflow Models - Deploy machine learning models in diverse serving environments MLflow Registry - Store, annotate, discover, and manage models in a central repository
  6. 6. MLFlow AutoML End-to-End ML Lifecycle ML Runtime and Environments Batch and Streaming Online Serving Data Science Workspace Prep Data Build Model Deploy/Monitor Model Open, pluggable architecture
  7. 7. MLFlow – Model Serving Prep Data Build Model Deploy/Monitor Model AutoML End-to-End ML Lifecycle Batch and Streaming Online Serving Data Science Workspace Open, pluggable architecture ML Runtime and Environments
  8. 8. Serving
  9. 9. MLflow Serving • Expose Mlflow model predictions as REST endpoint • Small cluster is automatically provisioned • HTTP endpoint publicly exposed • Limited production capability for now • For now, intended for light loads and testing
  10. 10. Model Serving from Model Registry Models Flavor 2 Flavor 1 Custom Models
  11. 11. Model Serving from Model Registry Models Tracking Flavor 2 Flavor 1 Custom Models Parameters Metrics Artifacts Models Metadata
  12. 12. Model Serving from Model Registry Staging Production Archived Data Scientists Deployment Engineers v1 v2 v3 Models Tracking Flavor 2 Flavor 1 Model Registry Custom Models Parameters Metrics Artifacts Models Metadata
  13. 13. Model serving from Model Registry Staging Production Archived Data Scientists Deployment Engineers v1 v2 v3 Models Tracking Flavor 2 Flavor 1 Model Registry Custom Models In-Line Code Containers Batch & Stream Scoring Cloud Inference Services OSS Serving Solutions Serving Parameters Metrics Artifacts Models Metadata
  14. 14. MLFlow Serving
  15. 15. Demo
  16. 16. Manage Served Versions • All active (non-archived) model versions are deployed • Manage model access rights • Source deployed model versions • Source via UI • Source via REST API request
  17. 17. Monitor Served Models • Displays status indicators for the serving cluster as well as individual model versions • Inspect the state of the serving cluster - displays a list of all serving events for this model • To inspect the state of a single model version
  18. 18. Customize Serving Cluster • Modify the memory size and number of cores of a serving cluster • Ability to add the tags • Ability to edit or delete an existing tags
  19. 19. Demo Code & Deck • https://github.com/KRSNagaraj/Mod elServing-DataAISummit2021
  20. 20. Q & A
  21. 21. Feedback Your feedback is important to us. Don’t forget to rate and review the sessions.

Productionzing ML Models are needs to ensure model integrity while it efficiently replicate runtime environments across servers besides it keep track of how each of our models were created. It helps us better trace the root cause of changes and issues over time as we acquire new data and update our model. We have greater accountability over our models and the results they generate. MLflow Model Serving delivers cost-effective and on-click deployment of model for real-time inferences. Also the Model Version deployed in the Model Serving can also be conveniently managed with MLflow Model Registry. We will going to cover following topics Deployment, Consumption and Monitoring. For deployment, we will demo the different version deployment and validate the deployment. For consumption, we demo connecting power bi and generate prediction report using ML Model deployed in MLflow serving. Lastly will wrap up with managing the MLflow serving like, access rights and monitoring capabilities.

Views

Total views

67

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

3

Shares

0

Comments

0

Likes

0

×