AI and machine learning (ML) can help solve complex problems. Azure Machine Learning allows developers to prepare data, train models in the cloud, and evaluate models. Windows ML enables using these trained models across Windows devices through APIs, with hardware acceleration for fast results. Developers can focus on their data and scenarios while Windows ML handles model evaluation. ONNX models can be used with Windows ML from tools like Azure Machine Learning and Custom Vision, which makes it easy to create and target models to platforms like iOS, Android, and Windows. Docker containers allow deploying these AI models to edge devices like Raspberry Pi for local inference.

1. AI on the Edge… and Containers
Riccardo Cappello
Microsoft MVP Microsoft Azure
Intel Software Innovator
Marco Dal Pino
Microsoft MVP Windows Development
Intel Software Innovator / Intel Blackbelt

2. why should I care about AI and ML?
As a developer,

3. Some problems are difficult to solve using traditional algorithms and
procedural programming.

4. AI solves hard problems
Smart Ink​
• Classify strokes as text, shapes,
freehand drawings​
• Classify text into words,
paragraphs, lines, bullets​
• Extract entities: phone numbers,
names, dates​
• Assign meanings: date references,
known contacts

5. Prepare Data Build & Train Evaluate
Azure Databricks Azure Machine Learning
Quickly launch and scale Spark on demand
Rich interactive workspace and notebooks
Seamless integration with all Azure data
services
Broad frameworks and tools support:
TensorFlow, Cognitive Toolkit, Caffe2, Keras,
MxNET, PyTorch
In the cloud – on the edge
Docker containers
Get started with machine learning
Windows Machine Learning

6. ScalabilityLow latency Flexibility
AI on the edge

7. Windows ML

8. 1. Developers can focus on their data and their
scenarios, using Windows ML for model
evaluation
2. Enables using ML models trained with a diverse
set of toolkits
3. Hardware acceleration gets fast evaluation results
across the diversity of the entire Windows device
ecosystem.
Windows ML solves three problems for you
Direct3D
GPU
CPU
DirectML
Model Inference Engine
WinML Win32 API
WinML UWP API
Win32 App
WinML Runtime
UWP App

9. Windows ML uses ONNX models

10. • Azure Machine Learning Services gives you an end-to-end solution to prepare data, and train your
model in the Cloud.
• WinMLTools converts existing models from CoreML, scikit-learn, LIBSVM, and XGBoost
• Azure Custom Vision makes it easy to create your own image models - https://customvision.ai/
• Azure AI Gallery curates models for use with Windows ML - https://gallery.azure.ai/models
How do I get ONNX models to use in my
application?

11. Azure Custom Vision
Demo

12. More edges with Custom Vision

13. Custom Vision models targets
CoreML
iOS 11 & later
TensorFlow
Android
ONNX
Windows ML
Dockerfile
Azure IoT Edge, Azure Functions, AzureML

14. Custom Vision offline models with iOS
& Android
Demo

15. AI on Docker
Demo

16. AI on Raspeberry Pi with Docker
Demo

17. Modify files to make things work
Dockerfile: add the following
https://gist.github.com/marcominerva/
74ab33bef3660bb9a0f6db2f6cebd97b

18. References
• https://onnx.ai
• https://gallery.azure.ai/models
• Emoji8: https://www.microsoft.com/en-
us/store/p/emoji8/9n5vj4hkp6x1
• https://customvision.ai
• https://azure.microsoft.com/en-us/blog/running-cognitive-
service-containers/
• https://github.com/DotNetToscana/CustomVisionCompanion

19. Thanks!
Questions?
Riccardo Cappello
Microsoft MVP Microsoft Azure
Intel Software Innovator
Marco Dal Pino
Microsoft MVP Windows Development
Intel Software Innovator / Intel Blackbelt