Successfully reported this slideshow.
Your SlideShare is downloading. ×

MLOps Using MLflow

MLOps Using MLflow

Download to read offline

MLflow is an MLOps tool that enables data scientist to quickly productionize their Machine Learning projects. To achieve this, MLFlow has four major components which are Tracking, Projects, Models, and Registry. MLflow lets you train, reuse, and deploy models with any library and package them into reproducible steps. MLflow is designed to work with any machine learning library and require minimal changes to integrate into an existing codebase. In this session, we will cover the common pain points of machine learning developers such as tracking experiments, reproducibility, deployment tool and model versioning. Ready to get your hands dirty by doing quick ML project using mlflow and release to production to understand the ML-Ops lifecycle.

MLflow is an MLOps tool that enables data scientist to quickly productionize their Machine Learning projects. To achieve this, MLFlow has four major components which are Tracking, Projects, Models, and Registry. MLflow lets you train, reuse, and deploy models with any library and package them into reproducible steps. MLflow is designed to work with any machine learning library and require minimal changes to integrate into an existing codebase. In this session, we will cover the common pain points of machine learning developers such as tracking experiments, reproducibility, deployment tool and model versioning. Ready to get your hands dirty by doing quick ML project using mlflow and release to production to understand the ML-Ops lifecycle.

More Related Content

MLOps Using MLflow

  1. 1. MLOPs Using MLflow Nagaraj Sengodan Senior Technical Manager, HCL Technologies Nitin Raj Soundararajan Technical Consultant, Cognizant Worldwide Limited
  2. 2. Presenter Nagaraj Sengodan is a Senior Technical Manager in Data and Analytics Practice at HCL Technologies. Nitin Raj Soundararajan is a technical consultant focusing on advanced data analytics, data engineering, cloud scale analytics and data science.
  3. 3. Agenda MLOps MLOps Process and stages. Challenges MLFlow Components Mapping with MLOps stages Demo Real-world example for mlflow Q & A
  4. 4. MLOps
  5. 5. Why
  6. 6. Challenges Reproducibility in ML models ML Operations Model Management Tools and Technologies
  7. 7. MLOps Software took months to release; now it releases daily. How we deploy software From curve fitting to neural networks… How we use data + maths to train models Evolution of Software Engineering from punch card to Distributed version control… Github! How we write software Write Software Engineering Deploy DevOps Train ML Engineer MLOps
  8. 8. Collaboration in Nature Data Source PREPARE DATA Data Engineer Data Repository TRAIN MODEL Modelling Pipeline FEATURE TRAIN EVALUATE RELEASE MODEL Release Pipeline DEPLOY APPROVE PROFILE Data Scientist Model Registry ML Engineer Data Pipeline VALIDATE PACKAGE Collect
  9. 9. MLOps – Life-Cycle ML Orchestration ML Health Business Impact Model Governance CI / CD Machine Learning Models Business Value Collaboration Collaboration MLOps Productionizing Machine Learning Models
  10. 10. Life Cycle Coding Unit Test Cases Peer –Review Approval Commit Test Release Prod Release Software Writing Code for specific requirement / functionality Writing Unit test cases for each functionality and block to cover all boundary condition Peer review the code and make sure code meets the standard and best practices Code is approved to check-in to repository / version control Approved code is merged with main / feature branch for release rollover. Release the feature / main branch for Testing. Move the Tested version to production.
  11. 11. Life Cycle Coding Unit Test Cases Peer –Review Approval Commit Test Release Prod Release Analyse Data Data Preparation Building Model Evaluate the Model Model Optimization Deploy Model Monitor and Re-Train Software ML
  12. 12. Life Cycle Coding Unit Test Cases Peer –Review Approval Commit Test Release Prod Release Analyse Data Data Preparation Building Model Evaluate the Model Model Optimization Deploy Model Monitor and Re-Train Software ML Cover Functional Requirements Optimize Metrics GOAL
  13. 13. Life Cycle Coding Unit Test Cases Peer –Review Approval Commit Test Release Prod Release Analyse Data Data Preparation Building Model Evaluate the Model Model Optimization Deploy Model Monitor and Re-Train Software ML Cover Functional Requirements Optimize Metrics Depends on the Code Depends on Data, Choice of algorithm, params, etc… GOAL QUALITY
  14. 14. Life Cycle Analyse Data Data Preparation Building Model Evaluate the Model Model Optimization Deploy Model Monitor and Re-Train Software ML Cover Functional Requirements Optimize Metrics Depends on the Code Depends on Data, Choice of algorithm, params, etc… GOAL QUALITY Mostly one tech stack Combinations of many libraries and tools TECHNOLOGY Coding Unit Test Cases Peer –Review Approval Commit Test Release Prod Release
  15. 15. Life Cycle Coding Unit Test Cases Peer –Review Approval Commit Test Release Prod Release Analyse Data Data Preparation Building Model Evaluate the Model Model Optimization Deploy Model Monitor and Re-Train Software ML Cover Functional Requirements Optimize Metrics Depends on the Code Depends on Data, Choice of algorithm, params, etc… GOAL QUALITY Mostly one tech stack Combinations of many libraries and tools TECHNOLOGY Works deterministically Changing based on data… OUTCOME
  16. 16. A platform for the Complete Machine Learning Lifecycle
  17. 17. MLFlow Components Machine Learning lifecycle MLflow Tracking - Record and query experiments: code, data, config, and results MLflow Projects - Package data science code in a format to reproduce runs on any platform MLflow Models - Deploy machine learning models in diverse serving environments MLflow Registry - Store, annotate, discover, and manage models in a central repository
  18. 18. Mlflow Workflow Models Tracking Model Registry V0 V1 V1 V2 V3
  19. 19. Mlflow Tracking Tracking Server Notebooks Local Apps Cloud Jobs UI API Spark Data Source Parameters Metrics Artifacts Metadata Models
  20. 20. Mlflow Projects Project Spec Local Execution Remote Execution Code Config Metadata
  21. 21. Mlflow Models
  22. 22. Mlflow Registry
  23. 23. Demo # Enable MLflow Autologging mlflow.keras.autolog() X, y = get_training_data()opt = keras.optimizers.Adam(lr=params["learning_rate"], beta_1=params["beta_1"], beta_2=params["beta_2"], epsilon=params["epsilon"] model = Sequential()model.add(Dense(int(params["units"]), ...) model.add(Dense(1)) model.compile(loss="mse", optimizer=opt) rest = model.fit(X, y, epochs=50, batch_size=64, validation_split=.2)
  24. 24. What Next? To get started with mlflow, just pip install mlflow Docs and Tutorials mlflow.org Session materials and demo https://github.com/krsnagaraj/dataaisummit-mlflow Connect with us ▪ https://uk.linkedin.com/in/nagarajsengodan ▪ https://www.linkedin.com/in/nitinrajs
  25. 25. Thank You
  26. 26. Feedback Your feedback is important to us. Don’t forget to rate and review the sessions.

×