[AI] ML Operationalization with Microsoft Azure

ML Operationalization with Microsoft Azure
Kyle Akepanidtaworn (Krid/Kyle)
Cloud Solution Architect (Advanced Analytics & AI)
Global Digital Transformation Partnerships Team
Email | Linkedin | Quora | Medium | GitHub

Learning Objectives
In this workshop, you will learn:
1. To orchestrate and deploy updates to the application and the deep learning model in a unified way.
2. To design a DevOps pipeline to coordinate retrieving the latest best model from the model registry, packaging the
web application, deploying the web application and inferencing web service. You will also learn how to monitor the
model's performance after it is deployed so your startup can be proactive with performance issues.
The target audience for the workshop includes:
• Data Scientists
• App Developers
• AI Engineers
• DevOps Engineers
Copyright (c) Microsoft Corporation. All rights reserved. @Korkrid Akepanidtaworn, GSMO Digital Transformation Partnerships Team

{1.Let’s PlayandInteract}

https://aka.ms/AA627pi

{2.Introduction ofMLOps}

Machine Learning Across Microsoft
Microsoft 365

Machine Learning at Scale
Monthly active Office
365 users, leveraging
Office AI capabilities
180
Million
Questions being
asked on Cortana
18
Billion
Number of signals
analyzed on behalf of
users to identify patterns
of emerging threats daily
6.5
Trillion

But ML is hard!

Building
a model
Data ingestion Data analysis
Data
transformation
Data validation Data splitting
Trainer
Model
validation
Training
at scale
LoggingRoll-out Serving Monitoring
Machine Learning Process

E2E ML lifecycle
Train Model Validate Model Deploy ModelPackage Model Monitor Model
Retrain Model

MLOps = ML + DEV + OPS
Experiment
Data Acquisition
Business Understanding
Initial Modeling
Develop Operate
Continuous Delivery
Data Feedback Loop
System + Model Monitoring
ML
Modeling + Testing
Continuous Integration
Continuous Deployment

App developer
using Azure DevOps
Build appCollaborate Test app Release app Monitor app
Model reproducibility Model retrainingModel deploymentModel validation
Data scientist using
Azure Machine Learning

Code, dataset, and
environment versioning
App developer
using Azure DevOps

Automated ML
ML Pipelines
Hyperparameter tuning
Train model
App developer
using Azure DevOps

Model validation
& certification
Train model Validate
model
App developer
using Azure DevOps

Model packaging
Simple deployment
model
Deploy
model
App developer
using Azure DevOps

Model
management
& monitoring
Model performance
analysis
model
Deploy
model
Monitor
model
Retrain model
App developer
using Azure DevOps

model
Deploy
model
Monitor
model
Retrain model
Azure DevOps integration
App developer
using Azure DevOps

MLOps Benefits
1. Reproducibility + Auditability
• Code drives generation and
deployments
• Pipelines are reproducible and verifiable
• All artifacts can be tagged and audited
2. Validation
• SWE best practices for quality control
• Offline comparisons of model quality
• Minimize bias and enable explainability
3. Automation + Observability
• Controlled rollout capabilities.
• Live comparison of predicted vs.
expected performance.
• Results fed back to watch for drift and
improve model.

Azure MLOps
Asset management & orchestration
services to help manage the ML lifecycle.

{3.Hands-on Workshop}

https://aka.ms/AA610wt

{4.Before Hands-on Lab}

Critical Requirements
1. Azure subscription. You will need a valid and active Azure account to complete the quickstarts. If you do
not have one, you can sign up for a free trial.
• The Microsoft Azure subscription must be pay-as-you-go or MSDN.
• Trial subscriptions will not work. You will run into issues with Azure resource quota limits.
• Subscriptions with access limited to a single resource group will not work. You will need the ability to deploy
multiple resource groups.
2. Azure DevOps subscription. You will need a valid and active Azure DevOps account to complete the
quickstarts. If you do not have one, you can sign up for a free account. (Please provision in “East US” to
leverage GPU.)
3. Azure Notebooks. You will need an Azure Notebooks project to import the quickstart notebooks into. See
instructions below on how to prepare your Azure Notebooks environment. (Please provision in “East
US” to leverage GPU.)
4. Azure Machine Learning service workspace. (Please provision in “East US” to leverage GPU.)

https://azure.com/devops
@AzureDevOps
https://aka.ms/AzureDevOpsForum
https://aka.ms/DevOpsBlog/
Azure DevOps
#AzureDevOps

Azure DevOps
Deliver value to your users faster
using proven agile tools to plan,
track, and discuss work across
your teams.
Build, test, and deploy with CI/CD that
works with any language, platform,
and cloud. Connect to GitHub or any
other Git provider and deploy
continuously.
Get unlimited, cloud-hosted
private Git repos and collaborate
to build better code with pull
requests and advanced file
management.
Test and ship with confidence
using manual and exploratory
testing tools.
Create, host, and share packages with
your team, and add artifacts to your
CI/CD pipelines with a single click.
Azure Boards Azure ReposAzure Pipelines
Azure Test Plans Azure Artifacts
https://azure.com/devops
➔

Cloud-hosted pipelines for Linux, Windows and
macOS, with unlimited minutes for open source
Azure Pipelines
Any language, any platform, any cloud
Build, test, and deploy Node.js, Python,  Java, PHP, Ruby,
C/C++, .NET, Android, and iOS apps. Run in parallel on
Linux, macOS, and Windows. Deploy to Azure, AWS,
GCP or on-premises
Extensible
Explore and implement a wide range of community-
built build, test, and deployment tasks, along with
hundreds of extensions from Slack to SonarCloud.
Support for YAML, reporting and more
Best-in-class for open source
Ensure fast continuous integration/continuous delivery
(CI/CD) pipelines for every open source project. Get
unlimited build minutes for all open source projects with
up to 10 free parallel jobs across Linux, macOS and
Windows
https://azure.com/pipelines➔
Containers and Kubernetes
Easily build and push images to container registries like
Docker Hub and Azure Container Registry. Deploy
containers to individual hosts or Kubernetes.

Track work with Kanban boards, backlogs, team
dashboards, and custom reporting
Azure Boards
https://azure.com/devops➔
Connected from idea to release
Track all your ideas at every development stage and
keep your team aligned with all code changes linked
directly to work items.
Scrum ready
Use built-in scrum boards and planning tools to help
your teams run sprints, stand-ups, and planning
meetings.
Project insights
Gain new insights into the health and status of your
project with powerful analytics tools and dashboard
widgets.

Unlimited private Git repo hosting and support for
TFVC that scales from a hobby project to the
world’s largest Git repositories
Azure Repos
Works with your Git client
Securely connect with and push code into your Git
repos from any IDE, editor, or Git client.
Web hooks and API integration
Add validations and extensions from the marketplace
or build your own using web hooks and REST APIs.
Semantic code search
Quickly find what you’re looking for with code-aware
search that understands classes and variables.

Get end-to-end traceability. Run tests and log
defects from your browser. Track and assess quality
throughout your testing lifecycle.
Azure Test Plans
Capture rich data
Capture rich scenario data as you execute tests to
make discovered defects actionable. Explore user
stories without test cases or test steps. You can
create test cases directly from your exploratory test
sessions.
Test across web and desktop
Test your application where it lives. Complete
scripted tests across desktop or web scenarios. Test
on-premises application from the cloud and vice-
versa.
Get end-to-end traceability
Leverage the same test tools across your engineers
and user acceptance testing stakeholders. Pay for the
tools only when you need them.

Create and share Maven, npm, and NuGet package
feeds from public and private sources – fully
integrated into CI/CD pipelines
Azure Artifacts
Manage all package types
Get universal artifact management for Maven, npm,
and NuGet.
Add packages to any pipeline
Share packages, and use built-in CI/CD, versioning,
and testing.
Share code efficiently
Easily share code across small teams and large
enterprises.

Azure DevOps Services Pricing
Free
Unlimited users and build time
• Azure Pipelines: 10 parallel jobs with
unlimited minutes for CI/CD
• Azure Boards: Work item tracking and
Kanban boards
• Azure Repos: Unlimited public Git repos
Free
Start free with up to 5 users
• Azure Pipelines: Run 1 Microsoft-hosted
job for 1,800 minutes per month and 1
self-hosted job for any amount of time
Kanban boards
• Azure Artifacts: package management
• Unlimited stakeholders
Starts at $6
per user, per month for Boards & Repos*
Easy pricing that grows with your team
• Azure Pipelines: Run 1 Microsoft-hosted
job for 1,800 minutes per month and 1
self-hosted job for any amount of time
Kanban boards
• Azure Artifacts: package management
• Unlimited stakeholders
• Boards & Repos included for Visual
Studio subscribers
https://azure.com/pricing/details/devops/➔
5 Boards & Repos users and 5 Artifacts users free. Pipelines
with unlimited minutes, Test Plans users and additional
Artifacts users also available. Please see the Azure pricing
calculator for details.
*

{5.Hands-on LabBefore Lunch Break}

Registering the Model (via Notebook & Portal)

Setup New Project in Azure DevOps

Import Quickstart code from a GitHub Repo

Update the build YAML file

Create New Service Connection

Setup and Run the Build Pipeline

{6.Hands-on LabAfterLunch Break}

Review Build Outputs

Setup the Release Pipeline

Add the Build Artifact

Add Variables to Deploy & Test stage

Setup Agent Pool for Deploy & Test Stage

Add Use Python Version Task

Add Install Requirements Task

Add Deploy & Test Webservice task

Define Deployment Trigger

Enable Continuous Deployment Trigger

Save the Release Pipeline

{7.Test, Build, ReleasePipelines}

Test, Build, Release Pipelines

Test, Build, Release Pipelines (~1 Hour+)

{8.Q&A}

[AI] ML Operationalization with Microsoft Azure

More Related Content

What's hot

Similar to [AI] ML Operationalization with Microsoft Azure

More from Korkrid Akepanidtaworn

Recently uploaded

[AI] ML Operationalization with Microsoft Azure