Download as PDF, PPTX
![[DSC Europe 25] Stefan Brankovic - #ResumeIsDead. AI-Powered Interviews and C...](https://cdn.slidesharecdn.com/ss_thumbnails/qnmbsv0xq3uysdrq3sev-2-stefan-brankovic-job-bolt-260114111931-a065aa3d-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Slobodan Dolinic - Smart and Intelligent Green Region.pptx](https://cdn.slidesharecdn.com/ss_thumbnails/0bribinjsp6ghwtvsvor-2-sigre-slobodan-dolinic-260115093812-c9c10e90-thumbnail.jpg?width=640&height=640&fit=bounds)
Machine learning operations model book mlops
- 1. Prof. Dr. JanKirenz Machine Learning Operations (MLOps) Usage of Pipelines in the ML Lifecycle with Tensor Flow Extended (TFX) and Kubeflow Prof. Dr. Jan Kirenz HdM Stuttgart
- 2. Prof. Dr. JanKirenz 80-85% PoC Factory The Proof of Concept Factory Most companies... ● … conduct AI experiments and pilots but achieve a low scaling success rate ● … have significant under investment, yielding low returns Source: Accenture (2019) https://www.accenture.com/us-en/insights/artificial-intelligence/ai-investments
- 3. Prof. Dr. JanKirenz https://www.gartner.com/smarterwithgartner/gartner-top-10-data-and-analytics-trends-for-2021/ Scalable AI
- 4. Prof. Dr. JanKirenz ML Project Code The problem with scaling AI: ML code is only a fraction of a production-ready ML project code ML Code 5-10%
- 5. Prof. Dr. JanKirenz Monitoring Hidden technical debt in machine learning systems Sculley, D. et al. (2015). Hidden technical debt in machine learning systems. Advances in neural information processing systems, 28, pp. 2503-2511 Data Collection Configuration Feature Engineering Data Verification Metadata Management Model Analysis Serving Infra- structure Automation Process Management Machine Resource Management Testing and Debugging ML Code
- 6. Prof. Dr. JanKirenz Machine learning operations (MLOps) ● ML Engineering culture and practice that aims at unifying ML System development (Dev) and ML system operations (Ops) ● Tools and principles to support workflow standardization and automation through the ML system lifecycle (e.g. with pipelines)
- 7. Prof. Dr. JanKirenz Prof. Dr. Jan Kirenz Machine learning lifecycle
- 8. Prof. Dr. JanKirenz Plan Model Deployment Data Business Analyst Data Engineer Software Developer Data Scientist Lifecycle of an ML System Plan | Data | Model | Deployment
- 9. Prof. Dr. JanKirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Lifecycle of an ML System Plan | Data | Model | Deployment
- 10. Prof. Dr. JanKirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Data splitting Anomaly detection Data preprocessing Lifecycle of an ML System Plan | Data | Model | Deployment
- 11. Prof. Dr. JanKirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Evaluate model Model Training & tuning Select algorithm Data splitting Anomaly detection Data preprocessing Lifecycle of an ML System Plan | Data | Model | Deployment
- 12. Prof. Dr. JanKirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Retrain triggers Evaluate model Model Training & tuning Monitor model Select algorithm Data splitting Anomaly detection Data preprocessing Lifecycle of an ML System Plan | Data | Model | Deployment
- 13. Prof. Dr. JanKirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Retrain triggers Evaluate model Model Training & tuning Monitor model Select algorithm Data splitting Anomaly detection Data preprocessing Lifecycle of an ML System Plan | Data | Model | Deployment Common issues which lead to a PoC to production gap ● Lack of reuse and duplication ● Inconsistency (data, code, models) ● Manual and slow transition from PoC to production
- 14. Prof. Dr. JanKirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Retrain triggers Evaluate model Model Training & tuning Monitor model Select algorithm Data splitting Anomaly detection Data preprocessing Model management Model registry Data and feature management Feature store Pipeline management Pipeline orchestration Metadata management Metadata store Lifecycle of an ML System Plan | Data | Model | Deployment MLOps components
- 15. Prof. Dr. JanKirenz What is a pipeline? ● Description of an ML workflow ● A pipeline component is a self-contained set of user code that performs one step in the pipeline ● Includes the definition of the configuration and inputs required to run the pipeline (e.g. model hyperparameters) … do this … than that Start ... … the end The workflow is also called directed acyclic graph (DAG) This is a component Complete workflow of the ML system lifecycle
- 16. Prof. Dr. JanKirenz Source: Baer & Ngahane (2019) … do this … than that Start … the end
- 17. Prof. Dr. JanKirenz TensorFlow Extended (TFX) ● Google-production-scale machine learning (ML) platform based on TensorFlow ● Portable to multiple environments (Azure, AWS, Google Cloud, IBM, ...) ● Python based toolkit; can be used with notebooks ● Helps you orchestrate your ML process: Apache Airflow, Apache Beam or Kubeflow pipelines Source: TensorFlow (2021)
- 18. Prof. Dr. JanKirenz TFX 1.0 (19.05.21) ● Enterprise-grade support ● Security patches and select bug fixes for up to three years ● Guaranteed API & Artifact backward compatibility Source: Google (2021)
- 19. Prof. Dr. JanKirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Evaluate model Model Training & tuning Monitor model ExampleGen Select algorithm StatisticsGen SchemaGen Example Validator Transform Data splitting Trainer Tuner Evaluator InfraValidator Anomaly detection Pusher HUB / JS / LITE / SERVING Model Server TF Data Validation (TFDV) TFT TF KerasTuner TensorFlow Model Analysis (TFMA) BulkInferrer Data preprocessing Metadata Store: ML Metadata (MLMD) TFX Options for Pipeline Orchestration
- 20. Prof. Dr. JanKirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Evaluate model Model Training & tuning Monitor model ExampleGen Select algorithm StatisticsGen SchemaGen Example Validator Transform Data splitting Trainer Tuner Evaluator InfraValidator Anomaly detection Pusher HUB / JS / LITE / SERVING Model Server TF Data Validation (TFDV) TFT TF KerasTuner TensorFlow Model Analysis (TFMA) BulkInferrer Metadata Store (ML Metadata) TFX Options for Pipeline Orchestration Data preprocessing
- 21. Prof. Dr. JanKirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Evaluate model Model Training & tuning Monitor model ExampleGen Select algorithm StatisticsGen SchemaGen Example Validator Transform Data splitting Trainer Tuner Evaluator InfraValidator Anomaly detection Pusher HUB / JS / LITE / SERVING Model Server TF Data Validation (TFDV) TFT TF KerasTuner TensorFlow Model Analysis (TFMA) BulkInferrer Data preprocessing Metadata Store (ML Metadata) TFX Options for Pipeline Orchestration
- 22. Prof. Dr. JanKirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Evaluate model Model Training & tuning Monitor model ExampleGen Select algorithm StatisticsGen SchemaGen Example Validator Transform Data splitting Trainer Tuner Evaluator InfraValidator Anomaly detection Pusher HUB / JS / LITE / SERVING Model Server TF Data Validation (TFDV) TFT TF TensorFlow Model Analysis (TFMA) BulkInferrer Data preprocessing Metadata Store (ML Metadata) TFX Options for Pipeline Orchestration
- 23. Prof. Dr. JanKirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Evaluate model Model Training & tuning Monitor model ExampleGen Select algorithm StatisticsGen SchemaGen Example Validator Transform Data splitting Trainer Tuner Evaluator InfraValidator Anomaly detection Pusher HUB / JS / LITE / SERVING Model Server TF Data Validation (TFDV) TFT TF TensorFlow Model Analysis (TFMA) BulkInferrer Data preprocessing Metadata Store (ML Metadata) TFX Options for Pipeline Orchestration
- 24. Prof. Dr. JanKirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Evaluate model Model Training & tuning Monitor model ExampleGen Select algorithm StatisticsGen SchemaGen Example Validator Transform Data splitting Trainer Tuner Evaluator InfraValidator Anomaly detection Pusher HUB / JS / LITE / SERVING Model Server TF Data Validation (TFDV) TFT TF KerasTuner TensorFlow Model Analysis (TFMA) BulkInferrer Data preprocessing Metadata Store (ML Metadata) TFX Options for Pipeline Orchestration
- 25. Prof. Dr. JanKirenz Tuner Evaluator InfraValidator ExampleGen StatisticsGen SchemaGen Example Validator Transform Trainer Pusher HUB / JS / LITE / SERVING Model Server KerasTuner BulkInferrer Metadata Store (ML Metadata) TF Data Validation (TFDV) TFT TF TFX Options for Pipeline Orchestration TensorFlow Model Analysis (TFMA) TensorFlow Lite is a set of tools that enables on-device machine learning by helping developers run their models on mobile, embedded, and IoT devices.
- 26. Prof. Dr. JanKirenz Plan Model Deployment Data Identify use case Frame problem Identify variables Define metrics Business Analyst Data Engineer Software Developer Data Scientist Data ingestion Analyze & clean data Define schema Feature engineering Validate model Deploy model Serve model Retrain model Evaluate model Model Training & tuning Monitor model ExampleGen Select algorithm StatisticsGen SchemaGen Example Validator Transform Data splitting Trainer Tuner Evaluator InfraValidator Anomaly detection Pusher HUB / JS / LITE / SERVING Model Server TF Data Validation (TFDV) TFT TF KerasTuner TensorFlow Model Analysis (TFMA) BulkInferrer Data preprocessing Metadata Store (ML Metadata) TFX Options for Pipeline Orchestration Production phase: automate the execution of the ML pipeline based on a schedule or certain triggering conditions. Development phase: run the ML experiment, instead of manually executing each step. Data preparation phase: automatically ingest, validate and transform data and provide features to models
- 27. Prof. Dr. JanKirenz Prof. Dr. Jan Kirenz Pipeline orchestration
- 28. Prof. Dr. JanKirenz TFX & Apache Airflow ● Programmatically author, schedule and monitor workflows with Python code. ● User interface to visualize pipelines running in production, monitor progress, and troubleshoot issues.
- 29. Prof. Dr. JanKirenz TFX & Apache Beam ● Provides a framework for running batch and streaming data processing jobs that run on a variety of runners (Spark, Flink, ...). ● Beam provides an abstraction layer which enables TFX to run on any supported runner without code modifications ● TFX only uses the Beam Python API
- 30. Prof. Dr. JanKirenz TFX & Kubeflow pipelines The Kubeflow Pipelines platform consists of: ● An engine for scheduling multi-step ML workflows (using Kubernetes). ● User interface (UI) for managing and tracking experiments, jobs, and runs. ● Python SDK for defining and manipulating pipelines and components. ● Notebooks for interacting with the system using the SDK Kubeflow Pipelines is available as a core component of Kubeflow or as a standalone installation.
- 31. Prof. Dr. JanKirenz
- 32. Prof. Dr. JanKirenz
- 33. Prof. Dr. JanKirenz
- 34. Prof. Dr. JanKirenz KubeFlow
- 35. Prof. Dr. JanKirenz ML toolkit for Kubernetes
- 36. Prof. Dr. JanKirenz Google’s Vertex AI Launched in May 2021
- 37. Prof. Dr. JanKirenz ML Pipelines | wrap-up Source: TensorFlow (2021) By using a ML pipeline, you can: ● Automate your ML process, which lets you regularly retrain, evaluate, and deploy your model. ● Utilize distributed compute resources for processing large datasets and workloads. ● Increase the velocity of experimentation by running a pipeline with different sets of hyperparameters. To learn more visit the following tutorials @: https://kirenz.github.io/ MLOps tutorials on how to: ● Install TF and TFX ● Build your first TFX pipeline ● Install Kubeflow ● Build your first Kubeflow pipeline Jan Kirenz www.kirenz.com
- 38. Prof. Dr. JanKirenz Backup Continuous Integration and Continuous Delivery pipeline for an ML/AI project in Microsoft Azure
- 39. Prof. Dr. JanKirenz Continuous Integration and Continuous Delivery pipeline for an ML/AI project in Microsoft Azure
- 40. Prof. Dr. JanKirenz CI/CD and source code management
- 41. Prof. Dr. JanKirenz Cloud based machine learning service
- 42. Prof. Dr. JanKirenz Source: Microsoft (2021)https://github.com/Microsoft/MLOpsPython
- 43. Prof. Dr. JanKirenz Source: Microsoft (2021) https://docs.microsoft.com/en-us/azure/architecture/reference-architectures/ai/mlops-python 1 2 3
- 44. Prof. Dr. JanKirenz https://github.com/Microsoft/MLOpsPython
- 45. Prof. Dr. JanKirenz
- 46. Prof. Dr. JanKirenz
- 47. Prof. Dr. JanKirenz
- 48. Prof. Dr. JanKirenz
- 49. Prof. Dr. JanKirenz
- 50. Prof. Dr. JanKirenz
- 51. Prof. Dr. JanKirenz
- 52. Prof. Dr. JanKirenz
- 53. Prof. Dr. JanKirenz End of Demo Continuous Integration and Continuous Delivery pipeline for an ML/AI project in Microsoft Azure