Improving Machine Learning  Workflows: Training, Packaging and Serving.

•

0 likes•423 views

The document discusses improving machine learning workflows by addressing struggles with manual processes, tracking experiments, packaging and versioning models, and model serving. It proposes using mlflow for tracking models and TensorFlow Serving for serving models. This will help establish continuous integration and delivery (CI/CD) of machine learning models to solve problems around reproducibility, deployment speed, and transparency.

Technology

Improving
Machine Learning 
Workflows
Training, Packaging and Serving.

We build speech and NLP models  
to help call centre agents
to better assist customers in real time.

Manual Work
Struggles
Copying data from server to
server
Training &
Experimentation
Keeping track of
experiments
Keeping track of artefacts
Reproducibility
Packaging & versioning
models
Manual deployments
No CI/CD
Longer release cycles
Slow model / code
integration
No transparency
Serving
Versioning
Model encapsulation

- Machine Learning Lifecycle.
- Building Blocks:  
mlflow & TensorFlow Serving.
- Infrastructure:  
Putting all components
together.
- CI/CD for ML models.
Agenda

Building blocks:  
mlflow & TensorFlow Serving

Solutions
Manual Work
Training &
Experimentation
No CI/CD Serving

Wilder Rodrigues
@wilderrodrigues
 
medium.com/@wilder.rodrigues
- Software & Artificial Intelligence Engineer
- Apache™ Committer & PMC member
- Keras contributor
- School of AI Utrecht Dean
- Amsterdam AI Ambassador
- Public Speaker

Similar to Improving Machine Learning  Workflows: Training, Packaging and Serving.

Getting machine learning models to production is notoriously difficult: it involves multiple teams (data scientists, data and machine learning engineers, operations, …), who often does not speak to each other very well; the model can be trained in one environment but then productionalized in completely different environment; it is not just about the code, but also about the data (features) and the model itself… At DataSentics, as a machine learning and cloud engineering studio, we see this struggle firsthand – on our internal projects and client’s projects as well.

Building a MLOps Platform Around MLflow to Enable Model Productionalization i...

Databricks

AML_service.pptx

Abhishek878239

Transformer-based pretrained language models such as BERT, XLNet, Roberta and Albert significantly advance the state-of-the-art of NLP and open doors for solving practical business problems with high performance transfer learning. However, operationalizing these models with production-quality continuous integration/ delivery (CI/CD) end-to-end pipelines that cover the full machine learning life cycle stages of train, test, deploy and serve while managing associated data and code repositories is still a challenging task.

Continuous Delivery of Deep Transformer-Based NLP Models Using MLflow and AWS...

Databricks

Transformer-based pre-trained language models such as BERT, XLNet, Roberta and Albert significantly advance the state-of-the-art of NLP and open doors for solving practical business problems with high-performance transfer learning. However, operationalizing these models with production-quality continuous integration/ delivery (CI/CD) end-to-end pipelines that cover the full machine learning life cycle stages of train, test, deploy and serve while managing associated data and code repositories is still a challenging task. In this presentation, we will demonstrate how we use MLflow and AWS Sagemaker to productionize deep transformer-based NLP models for guided sales engagement scenarios at the leading sales engagement platform, Outreach.io.

Continuous Delivery of Deep Transformer-Based NLP Models Using MLflow and AWS...

Databricks

MLOps in action

Pieter de Bruin

DevOps for Machine Learning overview en-us

eltonrodriguez11

Machine Learning para devs com ML.NET

Letticia Nicoli

Managing the Machine Learning Lifecycle with MLflow

Databricks

Successfully building a machine learning model is hard enough. Reproducing your results at scale — enabling others to reproduce pipelines, comparing results from other versions, moving models into production, redeploying and rolling out updated models — is exponentially harder. To address these challenges and accelerate innovation, many companies are building custom “ML platforms” to automate the end-to-end ML lifecycle. Watch a replay of this MLOps Virtual Event to hear more about the latest developments and best practices for managing the full ML lifecycle on Databricks with MLflow. We covered a checklist of capabilities you’ll need, common pitfalls, technological and organizational challenges, and how to overcome them. https://www.youtube.com/playlist?list=PLTPXxbhUt-YUFNBwBsSIlknoNbS7GExZw

MLOps Virtual Event | Building Machine Learning Platforms for the Full Lifecycle

Databricks

DevOps instincts tend to be shaped by what has worked well before. Instincts derived from mainstream software development projects get challenged when we turn to enabling machine learning projects. The key reasons are that the development/delivery workflow is different and the kind of software artefacts involved are different. We will explore the differences and look at emerging open source projects in order to appreciate why the DevOps for machine learning space is growing and the needs that it addresses.

Why is dev ops for machine learning so different

Ryan Dawson

Deployment Design Patterns - Deploying Machine Learning and Deep Learning Mod...

All Things Open

While the adoption of machine learning and deep learning techniques continue to grow, many organizations find it difficult to actually deploy these sophisticated models into production. It is common to see data scientists build powerful models, yet these models are not deployed because of the complexity of the technology used or lack of understanding related to the process of pushing these models into production. As part of this talk, I will review several deployment design patterns for both real-time and batch use cases. I’ll show how these models can be deployed as scalable, distributed deployments within the cloud, scaled across hadoop clusters, as APIs, and deployed within streaming analytics pipelines. I will also touch on topics related to security, end-to-end governance, pitfalls, challenges, and useful tools across a variety of platforms. This presentation will involve demos and sample code for the the deployment design patterns.

AllThingsOpen 2018 - Deployment Design Patterns (Dan Zaratsian)

dtz001

Note: The Content was modified from the Microsoft Content team. Deck Owner: Nitah Onsongo Tech/Msg Review: Cesar De La Torre, Simon Tao, Clarke Rahrig --- Event: Insider Dev Tour Berlin Event Description: Microsoft is going on a world tour with the announcements of Build 2019. The Insider Dev Tour focuses on innovations related to Microsoft 365 from a developer's perspective. Date: June 7th, 2019 Event link: https://www.microsoft.com/de-de/techwiese/news/best-of-build-insider-dev-tour-am-7-juni-in-berlin.aspx Linkedin: http://linkedin.com/in/mia-chang/

Leverage the power of machine learning on windows

Mia Chang

For real-world ML systems, it is crucial to have scalable and flexible platforms to build ML workflows. In this workshop, we will demonstrate how to build an ML DevOps pipeline using Kubeflow and TensorFlow Extended (TFX). Kubeflow is a flexible environment to implement ML workflows on top of Kubernetes - an open-source platform for managing containerized workloads and services, which can be deployed either on-premises or on a Cloud platform. TFX has a special integration with Kubeflow and provides tools for data pre-processing, model training, evaluation, deployment, and monitoring. In this workshop, we will demonstrate a pipeline for training and deploying an RNN-based Recommender System model using Kubeflow. https://papislatam2019.sched.com/event/OV1M/training-and-deploying-ml-models-with-kubeflow-and-tensorflow-extended-tfx-sponsored-by-cit

PAPIs LATAM 2019 - Training and deploying ML models with Kubeflow and TensorF...

Gabriel Moreira

Talk presented at PAPIs LATAM 2019 by Gabriel Moreira, Rodrigo Pereira and Fabio Uechi, from CI&T Summary: For real-world ML systems, it is crucial to have scalable and flexible platforms to build ML workflows. In this workshop, we will demonstrate how to build an ML DevOps pipeline using Kubeflow and TensorFlow Extended (TFX). Kubeflow is a flexible environment to implement ML workflows on top of Kubernetes - an open-source platform for managing containerized workloads and services, which can be deployed either on-premises or on a Cloud platform. TFX has a special integration with Kubeflow and provides tools for data pre-processing, model training, evaluation, deployment, and monitoring. In our workshop, we will demonstrate a pipeline for training and deploying an RNN-based Recommender System model using Kubeflow.

PAPIs LATAM 2019 - Training and deploying ML models with Kubeflow and TensorF...

Gabriel Moreira

When it comes to Large Scale data processing and Machine Learning, Apache Spark is no doubt one of the top battle-tested frameworks out there for handling batched or streaming workloads. The ease of use, built-in Machine Learning modules, and multi-language support makes it a very attractive choice for data wonks. However bootstrapping and getting off the ground could be difficult for most teams without leveraging a Spark cluster that is already pre-provisioned and provided as a managed service in the Cloud, while this is a very attractive choice to get going, in the long run, it could be a very expensive option if it’s not well managed. As an alternative to this approach, our team has been exploring and working a lot with running Spark and all our Machine Learning workloads and pipelines as containerized Docker packages on Kubernetes. This provides an infrastructure-agnostic abstraction layer for us, and as a result, it improves our operational efficiency and reduces our overall compute cost. Most importantly, we can easily target our Spark workload deployment to run on any major Cloud or On-prem infrastructure (with Kubernetes as the common denominator) by just modifying a few configurations. In this talk, we will walk you through the process our team follows to make it easy for us to run a production deployment of our Machine Learning workloads and pipelines on Kubernetes which seamlessly allows us to port our implementation from a local Kubernetes set up on the laptop during development to either an On-prem or Cloud Kubernetes environment

Infrastructure Agnostic Machine Learning Workload Deployment

Databricks

MLops on Vertex AI Presentation (AI/ML).pptx

Knoldus Inc.

Scaling AI/ML with Containers and Kubernetes

Tushar Katarki

Machine Learning Operations & Azure

Erlangen Artificial Intelligence & Machine Learning Meetup

Why is dev ops for machine learning so different - dataxdays

Ryan Dawson

Similar to Improving Machine Learning  Workflows: Training, Packaging and Serving. (20)

Building a MLOps Platform Around MLflow to Enable Model Productionalization i...

AML_service.pptx

Continuous Delivery of Deep Transformer-Based NLP Models Using MLflow and AWS...

MLOps in action

DevOps for Machine Learning overview en-us

Machine Learning para devs com ML.NET

Managing the Machine Learning Lifecycle with MLflow

MLOps Virtual Event | Building Machine Learning Platforms for the Full Lifecycle

Why is dev ops for machine learning so different

Deployment Design Patterns - Deploying Machine Learning and Deep Learning Mod...

AllThingsOpen 2018 - Deployment Design Patterns (Dan Zaratsian)

Leverage the power of machine learning on windows

PAPIs LATAM 2019 - Training and deploying ML models with Kubeflow and TensorF...

Infrastructure Agnostic Machine Learning Workload Deployment

MLops on Vertex AI Presentation (AI/ML).pptx

Scaling AI/ML with Containers and Kubernetes

Machine Learning Operations & Azure

Why is dev ops for machine learning so different - dataxdays

More from Wilder Rodrigues

The spread of Artificial Intelligence, mostly in the form of representational learning, has introduced an issue that, at first glance, seems difficult to get fixed: bias. Throughout the last years, more and more examples have emerged where people got mistaken by animals, bots became racists and machine learning solutions, chiefly used by recruitment companies and translation services, spot on the news with bias related issues. In this presentation, we will see the impact bias has on people and how to fix it without having to dive deep into the data and remove it manually.

Neutralising bias on word embeddings

Wilder Rodrigues

In this presentation I go over the theory and practical aspects of applying Deep Learning to solve NLP problems, more specifically, developing models for sentiment analysis. All the code used in the demo can be found here: https://github.com/ekholabs/DLinK https://github.com/ekholabs/automated_ml The presentation is available on YouTube: https://www.youtube.com/watch?v=eZavheF5TBE I start on 1:06:02.

Deep Learning for Natural Language Processing

Wilder Rodrigues

Exploring Artificial Intelligence and in practice, with several network architectures implemented on the fly in a live coding session. In this talk we went through Shallow / Intermediate / Deep / Convolutional and Residual Networks. Showing the differences in hyperparameters tuning, which to use and when. In addition, also how they came to life though the mind of computer scientists like Geoffrey Hinton, Yoshua Bengio, Xavier Glorot and Adam Coates. The code presented in this talk can be found here: https://github.com/ekholabs/DLinK

Ai - A Practical Approach

Wilder Rodrigues

This presentation, inspired by the Lord of the Rings trilogy, depicts one of the most anticipated features of the JDK 9: Project Jigsaw. It covers the following topics: Architecture; Accessibility; Modularisation; Implied Readability; Restricted Readability; Fat JARs; and Java Images. Al the source code, inclusive a extensive documentation, can be found here: https://github.com/ekholabs/postcode-modules

Java 9: Jigsaw Project

Wilder Rodrigues

Microservices with Spring Cloud

Wilder Rodrigues

Machine intelligence

Wilder Rodrigues

This talk is about going back to the basics, when Reactive Programming was not a buzz and not used in the Enterprise Arena. There will be moments when you will regret being here, but then we will present you with the Beauty of the Seventies. This talk intends to make you travel through time, experiencing how things were done back then. From the Beauty of the Seventies we will move straight to the GoF Observer pattern, which has been implemented over and over again with Publish / Subscriber mechanisms. That will depict the foundations of what we have seen with Functional Programming, Reactive Programming, FRP and Reactive Streams. Although all that sounds interesting, it’s not our last stop. We still have to go over Java 9 and Spring Web Reactive Framework. It embraces Reactive Streams and Reactor, by defying the challenge of switching from imperative-style to non-blocking logic. It might be seem by some as shiny and new, but it’s based on a well established idea that has been helping us for more than 40 years.

Embracing Reactive Streams with Java 9 and Spring 5

Wilder Rodrigues

More from Wilder Rodrigues (7)

Neutralising bias on word embeddings

Deep Learning for Natural Language Processing

Ai - A Practical Approach

Java 9: Jigsaw Project

Microservices with Spring Cloud

Machine intelligence

Embracing Reactive Streams with Java 9 and Spring 5

Recently uploaded

The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.

Search and Society: Reimagining Information Access for Radical Futures

Bhaskar Mitra

Join us as we dive into the latest updates to the UiPath Orchestrator API, including new limits and features for 2024. Discover how these changes can enhance your automation projects and streamline your workflows. 📚 Overview of UiPath Orchestrator API 🔧 Recent changes to API limits 🛠️ How to adapt to new limits 📋 Best practices for using the Orchestrator API efficiently ❓ Q&A session

Exploring UiPath Orchestrator API: updates and limits in 2024 🚀

DianaGray10

ScyllaDB has the potential to deliver impressive performance and scalability. The better you understand how it works, the more you can squeeze out of it. But before you squeeze, make sure you know what to monitor! Watch our experienced Postgres developer work through monitoring and performance strategies that help him understand what mistakes he’s made moving to NoSQL. And learn with him as our database performance expert offers friendly guidance on how to use monitoring and performance tuning to get his sample Rust application on the right track. This webinar focuses on using monitoring and performance tuning to discover and correct mistakes that commonly occur when developers move from SQL to NoSQL. For example: - Common issues getting up and running with the monitoring stack - Using the CQL optimizations dashboard - Common issues causing high latency in a node - Common issues causing replica imbalance - What a healthy system looks like in terms of memory - Key metrics to keep an eye on This isn’t “Death-by-Powerpoint.” We’ll walk through problems encountered while migrating a real application from Postgres to ScyllaDB – and try to fix them live as well.

Optimizing NoSQL Performance Through Observability

ScyllaDB

New customer? New industry? New cloud? New team? A lot to handle! How to ensure the success of the project? Start it well! I've created the 3 areas of focus at the beginning of the project that helped me in multiple roles (BA, PO, and Consultant). Learn from real-world experiences and discover how these insights can empower you to deliver unparalleled value to your customers right from the project's start.

Powerful Start- the Key to Project Success, Barbara Laskowska

CzechDreamin

How to differentiate Sales Cloud and CPQ on first glance might be tricky if you do not know where to look and what to look at. You will know :-) Managing the sales process within Salesforce is a common use case that can be managed with standart Sales Cloud. If you want to do entire quoting process you will find out Salesforce CPQ solution exists. What is then the difference if both can handle selling products? You will see comparison of 10 different features, which Sales Cloud and Salesforce CPQ handle differently. Simple question you will always remember if you should consider using Salesforce CPQ will be a cherry on top.

10 Differences between Sales Cloud and CPQ, Blanka Doktorová

CzechDreamin

Unlock the mysteries of successful Salesforce interviews in this insightful session hosted by Hugo Rosario (Salesforce Customer), a seasoned hiring manager that leads the Salesforce Department of multinational company with over 100 interviews under their belt. Step into the manager's chair and gain exclusive behind-the-scenes insights into what makes a Salesforce consultant stand out during the interview process. From deciphering the unspoken cues to mastering key strategies, we'll explore the intricacies of the interview process and provide practical tips for consultants looking to not only pass interviews but also thrive in their roles. Whether you're a seasoned professional or just starting your Salesforce journey, this session is your backstage pass to the secrets that hiring managers wish you knew.

Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...

CzechDreamin

Key Trends Shaping the Future of Infrastructure.pdf

Cheryl Hung

De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...

Product School

💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™: See how to accelerate model training and optimize model performance with active learning Learn about the latest enhancements to out-of-the-box document processing – with little to no training required Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath. Speakers: 👨‍🏫 Andras Palfi, Senior Product Manager, UiPath 👩‍🏫 Lenka Dulovicova, Product Program Manager, UiPath

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

UiPathCommunity

We're living the AI revolution and Salesforce is adapting and bring new value to their customers. Einstein products are evolving rapidly and navigating their limitations, language support, and use cases can be challenging. Let's make review of what Einstein product are available currently, what are the capabilities and what can be used for in CEE region and how Rossie.ai can help to learn Salesforce speak Czech. We will explore the Einstein roadmap and I will make a short live demo (based on your vote) of some Einstein feature.

AI revolution and Salesforce, Jiří Karpíšek

CzechDreamin

Welcome to UiPath Test Automation using UiPath Test Suite series part 2. In this session, we will cover API test automation along with a web automation demo. Topics covered: Test Automation introduction API Example of API automation Web automation demonstration Speaker Pathrudu Chintakayala, Associate Technical Architect @Yash and UiPath MVP Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP

UiPath Test Automation using UiPath Test Suite series, part 2

DianaGray10

Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)

Julian Hyde

Ever caught yourself nodding along when someone mentions "delivering value" in Agile, but secretly wondering what the heck they actually mean? You're not alone! Join us for an eye-opening session where we'll strip away the buzzwords and dive into the heart of Agile—value delivery. But what is "value"? Is it a mythical unicorn in the world of software development, or is there more to this overused term? This isn't going to be a sit-and-get lecture. We're talking about a face-to-face, interactive meetup where YOU play a crucial role. Come along to: Define It: What does "value" really mean? We’ll build a definition that’s not just words, but a compass for your Agile journey. Contextualise It: Discover what value means specifically to you, your team, your company, and your industry. Because one size does not fit all. Deliver It: Share strategies and gather new ones for uncovering and delivering true value—no more shooting in the dark!

Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx

David Michel

From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...

Product School

IoT Analytics Company Presentation May 2024

IoTAnalytics

From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...

Product School

You’ve heard good data matters in Machine Learning, but does it matter for Generative AI applications? Corporate data often differs significantly from the general Internet data used to train most foundation models. Join me for a demo on building an open source RAG (Retrieval Augmented Generation) stack using Milvus vector database for Retrieval, LangChain, Llama 3 with Ollama, Ragas RAG Eval, and optional Zilliz cloud, OpenAI.

Introduction to Open Source RAG and RAG Evaluation

Zilliz

IESVE for Early Stage Design and Planning

IES VE

The standard Salesforce Approval process can be limiting in many ways, especially in complex scenarios. What if there was a way to implement very flexible approvals where one can use Apex code to make data updates in unrelated records, dynamically generate next steps details, and compute assignees on the fly? And still use UI-based configurations to implement concrete approval processes. In this session, we will share ideas behind such a solution and show a few lines of code to get you started.

Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder

CzechDreamin

Knowledge engineering: from people to machines and back

Elena Simperl

Recently uploaded (20)

Search and Society: Reimagining Information Access for Radical Futures

Exploring UiPath Orchestrator API: updates and limits in 2024 🚀

Optimizing NoSQL Performance Through Observability

Powerful Start- the Key to Project Success, Barbara Laskowska

10 Differences between Sales Cloud and CPQ, Blanka Doktorová

Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...

Key Trends Shaping the Future of Infrastructure.pdf

De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

AI revolution and Salesforce, Jiří Karpíšek

UiPath Test Automation using UiPath Test Suite series, part 2

Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)

Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx

From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...

IoT Analytics Company Presentation May 2024

From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...

Introduction to Open Source RAG and RAG Evaluation

IESVE for Early Stage Design and Planning

Custom Approval Process: A New Perspective, Pavel Hrbacek & Anindya Halder

Knowledge engineering: from people to machines and back