Slide deck used to introduce machine learning with Azure Machine Learning Service. Focus on deployment of models with the machine learning SDK and consumption of the models with Python and Go.

1. Focus on model deployment
AI with Azure Machine
Learning

2. Pre-Built AI
Azure Cognitive Services
Conversational AI
Azure Bot Service
Custom AI
Azure Machine Learning

4. Artificial Intelligence

5. 6

6. “Machine Learning is the field of study that
gives computers the ability to learn without
being explicitly programmed.”
-- Arthur Samuel, 1959

7. Types of machine learning
8
Supervised
Unsupervised
Semi supervised
Reinforcement learning

8. Supervised learning
▪ Training data includes the desired solution (called labels) and features
▪ E.g. predict house prices based on features of the house and neighborhood; you have a
data set that contains the house prices
▪ Classification (e.g. spam filter), regression (e.g. house price prediction)
9

9. Unsupervised learning
▪ Training data does not contain labels
10

10. 2016: 96%
accuracy
RESNET vision
test (152 layer
neural network)
2017: 5.1%
switchboard
speech
recognition test
Jan 2018: 88.5%
SQuAD reading
comprehension
test
March 2018:
69.9% MT
research system
11

11. Challenges
▪ Not enough data: thousands or millions of examples
required depending on the problem
▪ Data is not representative
▪ Sampling noise
▪ Sampling bias
▪ Poor quality of data: importance of “data cleansing”
▪ Errors
▪ Outliers
▪ Noise
▪ Irrelevant features
▪ Overfitting and underfitting
12

12. How does it work?
The “Hello World” of machine learning

13. Linear Regression? Just use Excel!
y=b + w * x

14. But how can we find the parameters b and w?
▪ Linear regression has a closed-form solution: the normal equation
▪ Calculates b and w directly
▪ Slow if there are many features
▪ Learn the parameters iteratively
▪ “Gradient Descent” algorithm
▪ Just select a random b and w and check
the cost
▪ Modify b and w to obtain a lower cost
and keep doing that for a number of times
15

15. Cost function
▪ Basically measures the difference between
the real values (blue) and predicted values
(red)
▪ There is a red line with the lowest cost
▪ The goal is to find that line by “cleverly”
trying new combinations of b and w
16

16. Gradient Descent
17

17. Learning rate: size of the steps you take
18

18. Not just for straight lines
19
y=w1 * x^2 + w2 * x + b

19. Demo
▪ Linear Regression in a Jupyter
Notebook
20

20. 21

21. Training models
Just the important bits to understand how models are built

22. Wait a moment! What’s Machine Learning?
▪ We will talk about supervised learning only
▪ Learn from labeled data
e.g. you have images of numbers and you know the actual number the image represents
e.g. you have images of cats and you know they are cats
▪ Make predictions on new samples
Learn a model
predict “label = 4” (based on the model)

23. Storage
Training a model (running experiments)
ComputeTrain.py

24. Python SDK
▪ Connect to your workspace
▪ Provision compute targets
▪ Upload data to storage
▪ Submit experiments
▪ Log training process
▪ Register models
▪ Deprovision compute targets
▪ Create container images
▪ Deploy as a web service
▪ ...

27. Training a model with train.py
▪ Load data from storage
▪ Fit ML model on the data
▪ use your framework of choice
▪ Save the model to storage
▪ Log metrics (e.g. accuracy) to Azure ML

28. Virtual
Machines
Azure ML
Compute
Kubernetes HDInsight
Azure
Databricks
Local
compute

29. Guided Exercise
▪ Log on to Azure Portal
▪ Create Azure ML service
▪ Log on to Azure Notebooks
▪ Perform the Getting Started exercise
(Monte Carlo simulation to calculate
Pi)
▪ uses local compute (of the
workbook)
30

30. Packaging Models
aka operationalizing models
aks run models in production

31. Storage
Packaging the model
ComputeTrain.py
Container
image
Score.py
Registered model

32. Scoring with score.py
▪ To make predictions (inference) you provide a score.py script
▪ Azure ML uses score.py to build an API you can deploy to computer targets
▪ The API is packaged in a Docker container
def init():
load(model)
def run(input)
prediction =
model(input)
return prediction

33. Storage
Custom models: Machine Learning Service
ComputeTrain.py
Container
image
Score.py
Registered model
Deploy ?

34. ONNX Models
We will use ONNX models and package them in a container

37. Why use ONNX?
▪ Better interoperability
▪ Spend less time to put models into production
▪ Ecosystem of tools for visualization and acceleration
▪ Converters
▪ Visualizers

38. Netron Visualizer

39. Guided Exercise
▪ Create a container image that uses
the ResNet50 model to classify
images
40

40. Deploying Models

41. Deploying models
▪ A model is a function to execute
▪ A model is deployed as a REST API endpoint
that you call from your custom application
▪ Inference = making decisions about new data
e.g. is it a cat?

42. Azure Container
Instances
Azure
Kubernetes
Service
Azure IoT Edge
Field-
programmable
gate array
(FPGA)

43. Deploy to Azure Container Instances (ACI)
▪ Simplest way to run
containers in Azure
▪ No need to think about
clusters
▪ Container groups with
multiple containers

44. Deploy to Azure Kubernetes Service (AKS)
▪ For high-scale production scenarios
▪ Use an existing AKS cluster or deploy from Azure ML
▪ More advanced capabilities
▪ Autoscaling
▪ Use of GPUs
▪ Monitoring and logging

46. GPUs on Kubernetes
▪ Deploy on GPU instances
▪ Ensure GPU capabilities are advertised
▪ use device plugins
▪ You need to specifically enable GPU
support in the container used for inference

47. Guided Exercise
▪ Deploying the ResNet50 model to
Azure Container Instances
▪ Inference with Python
▪ Inference with Go
50

48. On your own
▪ Deploying the FER+ (emotion
recognition for faces) model to Azure
Container Instances
▪ Inference with Python
51

49. customvision.ai
Transfer Learning Made Easy

50. Transfer Learning
Image from Medium

54. Predicting with the Prediction URL
▪ Post a URL or binary data to the prediction URL
▪ For binary data:

55. Export the model
▪ Set the domain to general compact: model will be smaller and potentially less accurate
▪ Export is now possible

56. Dockerfile
▪ Build the image with the included
Dockerfile:
▪ Push the image to a registry (e.g.
Docker Hub:
▪ docker push gbaeke/carmake

57. Run in ACI

58. Use container model

59. Running models at the edge
IoT Edge to the rescue

60. Deploy to IoT Edge
▪ IoT Edge provides the capability to
process telemetry at the network
edge
▪ Process data locally and forward to
the cloud when needed
▪ e.g. alert, run AI model, …
▪ IoT Edge uses modules and these
modules are containers
▪ Use IoT Hub to deploy an Azure
ML container to the edge device

61. Example of deployment to IoT Edge

62. FPGA

63. Deploy to FPGAs
▪ Array of programmable logic blocks and reconfigurable interconnects
▪ Configurable post-manufacturing
▪ Mainly used for inference (not for training)

64. Project Brainwave on Azure
▪ Based on Intel FPGA devices
▪ Available in Azure for real-time AI
(inference)
▪ Configurable for different types of pre-
trained deep neural networks (DNN)
▪ Image classification and recognition
▪ ResNet 50, ResNet 152, VGG-16, …
▪ High performance
> 500 classifications / second
▪ Not for your custom models

65. FPGA Speed Demo
68

66. care.grow.passion.