#insiderDevTour
Leverage the Power of
Machine Learning on Windows
Jose Antonio Silva
R&D Director at DevScope
#insiderDevTour
Why AI and ML?
How can you get started?
What else?
What we'll talk about today
#insiderDevTour
Examples
Classify if a customer is a retention risk
Identify objects in images and videos
Detect defects in an industrial process
Some problems are difficult to solve using traditional algorithms and
procedural programming.
These examples are good candidates for machine learning.
When can you use Machine Learning?
A repetitive decision
or process
Solution lacks an
explicit definition
A lot of training data
is available
#insiderDevTour
90%
Reduced time from
documents to insights
$300K
Cost savings per day
40x
More insurance cases reviewed
200K
Customers interacting
with chatbots
+20%
Increase in
customer satisfaction
+32%
Increase in sales
#insiderDevTour
#insiderDevTour
Prepare Your Data
Quickly launch and scale
Spark on demand
Rich interactive workspace
and notebooks
Seamless integration with all
Azure data services
Build and Train
Broad frameworks and tools
support
TensorFlow, Cognitive Toolkit,
Caffe2, Keras, MxNET, PyTorch
Deploy
Docker containers
Windows AI Platform
Azure Machine Learning
Machine Learning Landscape
#insiderDevTour
Machine Learning for Developers
Enable Multi-Input
Azure Cognitive Services
Ink Recognizer
Increase Functionality
Windows AI platform
Sentiment Analysis
ML.NET
Price prediction
#insiderDevTour
Machine Learning for Developers
Enable Multi-Input
Azure Cognitive Services
Ink Recognizer
Increase Functionality
Windows AI platform
Sentiment Analysis
ML.NET
Price prediction
#insiderDevTour
The most comprehensive pre-trained AI
Language
Vision
Speech
Decision
Web search
Bing Spell Check
Custom
Vision
Personalizer
Form Recognizer
Neural Text-to-Speech
Anomaly Detector Content
Moderator
Content Moderator
Custom Speech
Speech transcription
Text-to-Speech
Conversation
transcription capability
Face
Video
Indexer
Ink Recognizer
Computer
Vision Language
Understanding
QnA Maker
Text Analytics
Translator Text
Bing Web
Search
Bing Custom
Search
Bing
Video Search
Bing Image Search
Bing
Local Business
Search
Bing Visual Search
Bing Entity Search
Bing News
Search
Bing Autosuggest
The most comprehensive pre-trained AI
Ink Recognizer
Ink Recognizer
aka.ms/idt2019resources
#insiderDevTour
Demo
Ink recognizer
#insiderDevTour
Machine Learning for Developers
Enable Multi-Input
Azure Cognitive Services
Ink Recognizer
Increase Functionality
Windows AI platform
Sentiment Analysis
ML.NET
Price prediction
#insiderDevTour
DESKTOP CLOUDWEB MOBILE ML
.NET
Your platform for building anything
IoTGAMING
#insiderDevTour
ML.NET
Machine Learning framework for building custom ML Models
Proven at scale
Azure, Office, Windows
Extensible
TensorFlow, ONNX and Infer.NET
Cross-platform and open-source
Runs everywhere
Easy to use tools
CLI + UI-based tool for building models
#insiderDevTour
ML.NET Tooling
ML.NET CLI global tool accelerates productivity
AutoML
Model
Builder
#insiderDevTour
How much is the taxi fare for 1 passenger going from Cape Town to Johannesburg?
ML.NET CLI global tool accelerates productivity
#insiderDevTour
Criterion
Loss
Min Samples Split
Min Samples Leaf
XYZ
Parameter 1
Parameter 2
Parameter 3
Parameter 4
…
Distance
Trip time
Car type
Passengers
Time of day
…
Gradient Boosted
Nearest Neighbors
SGD
Bayesian Regression
LGBM
…
Distance Gradient Boosted
Model
Car type
Passengers
Getting started w/machine learning can be hard
ML.NET takes the guess work out of data prep, feature selection & hyperparameter tuning
Which algorithm? Which parameters?Which features?
#insiderDevTour
N Neighbors
Weights
Metric
P
ZYX
Which algorithm? Which parameters?Which features?
Distance
Trip time
Car type
Passengers
Time of day
…
Gradient Boosted
Nearest Neighbors
SGD
Bayesian Regression
LGBM
…
Nearest Neighbors
Criterion
Loss
Min Samples Split
Min Samples Leaf
XYZ Model
Iterate
Gradient BoostedDistance
Car brand
Year of make
Car type
Passengers
Trip time
Getting started w/machine learning can be hard
ML.NET takes the guess work out of data prep, feature selection & hyperparameter tuning
#insiderDevTour
Which algorithm? Which parameters?Which features?
Iterate
Getting started w/machine learning can be hard
ML.NET takes the guess work out of data prep, feature selection & hyperparameter tuning
#insiderDevTour
25%40%70%
25%
95%
25% 25%
25%
25%
40%
40%
40%
40%
70%
70%
70%Enter data
Define goals
Apply constraints
Input Intelligently test multiple models in parallel
Optimized model
95%
ML.NET accelerates model development
#insiderDevTour
70%95% Feature importance
Distance
Trip time
Car type
Passengers
Time of day
0 1
Model B (70%)
Distance
0 1
Trip time
Car type
Passengers
Time of day
Feature importance Model A (95%)
ML.NET accelerates model development
with model explainability
#insiderDevTour
Demo
Taxi fare demo
ML Model Builder
#insiderDevTour
# STEP 1: Load data
IDataView trainingDataView = mlContext.Data.LoadFromTextFile<TaxiTrip>( ... )
IDataView testDataView = mlContext.Data.LoadFromTextFile<TaxiTrip>( ... )
ConsoleHelper.ShowDataViewInConsole(mlContext, trainingDataView)
# STEP 2: Initialize user-defined progress handler that AutoML will invoke after each model
var progressHandler = new RegressionExperimentProgressHandler()
# STEP 3: Run AutoML regression experiment
ExperimentResult<RegressionMetrics> experimentResult = mlContext.Auto()
.CreateRegressionExperiment(ExperimentTime)
.Execute(trainingDataView, LabelColumnName, progressHandler: progressHandler)
PrintTopModels(experimentResult)
# STEP 4: Evaluate the model on test data
RunDetail<RegressionMetrics> best = experimentResult.BestRun
ITransformer trainedModel = best.Model
IDataView predictions = trainedModel.Transform(testDataView)
# STEP 5: Save trained model to a .ZIP file
mlContext.Model.Save(trainedModel, trainingDataView.Schema, ModelPath)
#insiderDevTour
Machine Learning for Developers
Enable Multi-Input
Azure Cognitive Services
Ink Recognizer
Increase Functionality
Windows AI platform
Sentiment Analysis
ML.NET
Price prediction
aka.ms/idt2019resources
#insiderDevTour
WinML
Practical, simple model-based API for ML
inferencing on Windows
DirectML
Realtime, high control ML operator API; part
of DirectX family
Compute Driver Model
Robust hardware reach/abstraction layer for
compute and graphics silicon
DirectML API
DirectX12
WinML API
ONNX Runtime
Compute Driver Model
GPU VPU xPU CPU
Windows AI platform
#insiderDevTour
# Load a model
var modelFile = await StorageFile.GetFileFromApplicationUriAsync(new Uri(modelPath))
LearningModel model = wait LearningModel.LoadFromStorageFileAsync(modelFile)
# Bind a model
LearningModelBinding binding = new LearningModelBinding(session)
ImageFeatureValue image = ImageFeatureValue.CreateFromVideoFrame(inputFrame)
binding.Bind(inputName, image);
# Evaluate
var result = await session.EvaluateAsync(binding, "0")
#insiderDevTour
✓ Azure Custom Vision makes it easy to create an image model
✓ WinMLTools converts existing models from TensorFlow, Keras, CoreML, scikit-learn, LIBSVM and
XGBoost
✓ Azure Machine Learning Service provides an end-to-end solution for preparing your data and
training your model
✓ ML.NET
Getting an ONNX Model
#insiderDevTour
WinML Benefits
• Low latency
• Cost Effectiveness
• Flexibility
DirectML API
DirectX12
WinML API
ONNX Runtime
Compute Driver Model
GPU VPU xPU CPU
Windows AI platform
Azure AI
Ink Recognizer
Windows AI platform
WinML
ML.NET
AutoML
What we’ve seen today…
…and two exciting tools announced at Build
DirectML
Vision Skills
✓Powers Windows ML hardware
acceleration
✓Delivers broad hardware
support through DirectX12
✓Built for real-time performance
✓Raytracing
DirectML API
DirectX12
WinML API
ONNX Runtime
Compute driver model
GPU VPU xPU CPU
What is DirectML?
Object detector Skeletal detector Sentiment analyzer
Vision Skills
#insiderDevTour
Why AI and ML?
✓ The future of Apps
Azure Cognitive Services
✓ Comprehensive pre-trained AI
✓ Many new updates!
ML.NET
✓ AutoML
✓ Model Builder
The Windows AI platform
✓ WinML
✓ DirectML
✓ Vision Skills
Recap
#insiderDevTour
Applications
Today
The Future of
Applications
AI
ML
The Power of Machine Learning
#insiderDevTour
Learn more!
Check out the labs and resources!
aka.ms/insiderdevtour-labs
#insiderDevTour
Insider Dev Tour

Leverage the power of machine learning on windows

  • 1.
    #insiderDevTour Leverage the Powerof Machine Learning on Windows Jose Antonio Silva R&D Director at DevScope
  • 2.
    #insiderDevTour Why AI andML? How can you get started? What else? What we'll talk about today
  • 3.
  • 4.
    Examples Classify if acustomer is a retention risk Identify objects in images and videos Detect defects in an industrial process Some problems are difficult to solve using traditional algorithms and procedural programming. These examples are good candidates for machine learning.
  • 5.
    When can youuse Machine Learning? A repetitive decision or process Solution lacks an explicit definition A lot of training data is available
  • 6.
    #insiderDevTour 90% Reduced time from documentsto insights $300K Cost savings per day 40x More insurance cases reviewed 200K Customers interacting with chatbots +20% Increase in customer satisfaction +32% Increase in sales
  • 7.
  • 8.
    #insiderDevTour Prepare Your Data Quicklylaunch and scale Spark on demand Rich interactive workspace and notebooks Seamless integration with all Azure data services Build and Train Broad frameworks and tools support TensorFlow, Cognitive Toolkit, Caffe2, Keras, MxNET, PyTorch Deploy Docker containers Windows AI Platform Azure Machine Learning Machine Learning Landscape
  • 9.
    #insiderDevTour Machine Learning forDevelopers Enable Multi-Input Azure Cognitive Services Ink Recognizer Increase Functionality Windows AI platform Sentiment Analysis ML.NET Price prediction
  • 10.
    #insiderDevTour Machine Learning forDevelopers Enable Multi-Input Azure Cognitive Services Ink Recognizer Increase Functionality Windows AI platform Sentiment Analysis ML.NET Price prediction
  • 11.
  • 12.
    The most comprehensivepre-trained AI Language Vision Speech Decision Web search Bing Spell Check Custom Vision Personalizer Form Recognizer Neural Text-to-Speech Anomaly Detector Content Moderator Content Moderator Custom Speech Speech transcription Text-to-Speech Conversation transcription capability Face Video Indexer Ink Recognizer Computer Vision Language Understanding QnA Maker Text Analytics Translator Text Bing Web Search Bing Custom Search Bing Video Search Bing Image Search Bing Local Business Search Bing Visual Search Bing Entity Search Bing News Search Bing Autosuggest
  • 13.
    The most comprehensivepre-trained AI Ink Recognizer
  • 14.
  • 15.
  • 16.
    #insiderDevTour Machine Learning forDevelopers Enable Multi-Input Azure Cognitive Services Ink Recognizer Increase Functionality Windows AI platform Sentiment Analysis ML.NET Price prediction
  • 17.
    #insiderDevTour DESKTOP CLOUDWEB MOBILEML .NET Your platform for building anything IoTGAMING
  • 18.
    #insiderDevTour ML.NET Machine Learning frameworkfor building custom ML Models Proven at scale Azure, Office, Windows Extensible TensorFlow, ONNX and Infer.NET Cross-platform and open-source Runs everywhere Easy to use tools CLI + UI-based tool for building models
  • 19.
    #insiderDevTour ML.NET Tooling ML.NET CLIglobal tool accelerates productivity AutoML Model Builder
  • 20.
    #insiderDevTour How much isthe taxi fare for 1 passenger going from Cape Town to Johannesburg? ML.NET CLI global tool accelerates productivity
  • 21.
    #insiderDevTour Criterion Loss Min Samples Split MinSamples Leaf XYZ Parameter 1 Parameter 2 Parameter 3 Parameter 4 … Distance Trip time Car type Passengers Time of day … Gradient Boosted Nearest Neighbors SGD Bayesian Regression LGBM … Distance Gradient Boosted Model Car type Passengers Getting started w/machine learning can be hard ML.NET takes the guess work out of data prep, feature selection & hyperparameter tuning Which algorithm? Which parameters?Which features?
  • 22.
    #insiderDevTour N Neighbors Weights Metric P ZYX Which algorithm?Which parameters?Which features? Distance Trip time Car type Passengers Time of day … Gradient Boosted Nearest Neighbors SGD Bayesian Regression LGBM … Nearest Neighbors Criterion Loss Min Samples Split Min Samples Leaf XYZ Model Iterate Gradient BoostedDistance Car brand Year of make Car type Passengers Trip time Getting started w/machine learning can be hard ML.NET takes the guess work out of data prep, feature selection & hyperparameter tuning
  • 23.
    #insiderDevTour Which algorithm? Whichparameters?Which features? Iterate Getting started w/machine learning can be hard ML.NET takes the guess work out of data prep, feature selection & hyperparameter tuning
  • 24.
    #insiderDevTour 25%40%70% 25% 95% 25% 25% 25% 25% 40% 40% 40% 40% 70% 70% 70%Enter data Definegoals Apply constraints Input Intelligently test multiple models in parallel Optimized model 95% ML.NET accelerates model development
  • 25.
    #insiderDevTour 70%95% Feature importance Distance Triptime Car type Passengers Time of day 0 1 Model B (70%) Distance 0 1 Trip time Car type Passengers Time of day Feature importance Model A (95%) ML.NET accelerates model development with model explainability
  • 26.
  • 27.
    #insiderDevTour # STEP 1:Load data IDataView trainingDataView = mlContext.Data.LoadFromTextFile<TaxiTrip>( ... ) IDataView testDataView = mlContext.Data.LoadFromTextFile<TaxiTrip>( ... ) ConsoleHelper.ShowDataViewInConsole(mlContext, trainingDataView) # STEP 2: Initialize user-defined progress handler that AutoML will invoke after each model var progressHandler = new RegressionExperimentProgressHandler() # STEP 3: Run AutoML regression experiment ExperimentResult<RegressionMetrics> experimentResult = mlContext.Auto() .CreateRegressionExperiment(ExperimentTime) .Execute(trainingDataView, LabelColumnName, progressHandler: progressHandler) PrintTopModels(experimentResult) # STEP 4: Evaluate the model on test data RunDetail<RegressionMetrics> best = experimentResult.BestRun ITransformer trainedModel = best.Model IDataView predictions = trainedModel.Transform(testDataView) # STEP 5: Save trained model to a .ZIP file mlContext.Model.Save(trainedModel, trainingDataView.Schema, ModelPath)
  • 28.
    #insiderDevTour Machine Learning forDevelopers Enable Multi-Input Azure Cognitive Services Ink Recognizer Increase Functionality Windows AI platform Sentiment Analysis ML.NET Price prediction aka.ms/idt2019resources
  • 29.
    #insiderDevTour WinML Practical, simple model-basedAPI for ML inferencing on Windows DirectML Realtime, high control ML operator API; part of DirectX family Compute Driver Model Robust hardware reach/abstraction layer for compute and graphics silicon DirectML API DirectX12 WinML API ONNX Runtime Compute Driver Model GPU VPU xPU CPU Windows AI platform
  • 30.
    #insiderDevTour # Load amodel var modelFile = await StorageFile.GetFileFromApplicationUriAsync(new Uri(modelPath)) LearningModel model = wait LearningModel.LoadFromStorageFileAsync(modelFile) # Bind a model LearningModelBinding binding = new LearningModelBinding(session) ImageFeatureValue image = ImageFeatureValue.CreateFromVideoFrame(inputFrame) binding.Bind(inputName, image); # Evaluate var result = await session.EvaluateAsync(binding, "0")
  • 31.
    #insiderDevTour ✓ Azure CustomVision makes it easy to create an image model ✓ WinMLTools converts existing models from TensorFlow, Keras, CoreML, scikit-learn, LIBSVM and XGBoost ✓ Azure Machine Learning Service provides an end-to-end solution for preparing your data and training your model ✓ ML.NET Getting an ONNX Model
  • 32.
    #insiderDevTour WinML Benefits • Lowlatency • Cost Effectiveness • Flexibility DirectML API DirectX12 WinML API ONNX Runtime Compute Driver Model GPU VPU xPU CPU Windows AI platform
  • 33.
    Azure AI Ink Recognizer WindowsAI platform WinML ML.NET AutoML What we’ve seen today… …and two exciting tools announced at Build DirectML Vision Skills
  • 34.
    ✓Powers Windows MLhardware acceleration ✓Delivers broad hardware support through DirectX12 ✓Built for real-time performance ✓Raytracing DirectML API DirectX12 WinML API ONNX Runtime Compute driver model GPU VPU xPU CPU What is DirectML?
  • 35.
    Object detector Skeletaldetector Sentiment analyzer Vision Skills
  • 36.
    #insiderDevTour Why AI andML? ✓ The future of Apps Azure Cognitive Services ✓ Comprehensive pre-trained AI ✓ Many new updates! ML.NET ✓ AutoML ✓ Model Builder The Windows AI platform ✓ WinML ✓ DirectML ✓ Vision Skills Recap
  • 37.
  • 38.
    #insiderDevTour Learn more! Check outthe labs and resources! aka.ms/insiderdevtour-labs
  • 39.