Azure Monitor & Application Insight to monitor Infrastructure & Application
Microsoft AI Platform - AETHER Introduction
1.
2.
3. Agenda
Scaled out AI Platform with Search (Bing and
Ads)
AI Platform is now used across Microsoft
Microsoft needs and drives Open and
Interoperable AI
Azure AI Services
4. Several Billion dollar AI businesses at
Microsoft
Learned a ton as we built them
Bing Ads Story
My AI Story
7. Bing Ads Execution
Shipped once every 6 months
3 experiments / month
Big bets that didn’t work.
Created dev teams with core metrics
Pushed teams to move their metric
Developed AI Infrastructure to do
rapid experimentation.
Over a few years made >300x
improvement (1000/mo expr)
8. Experiment – 90% of ideas fail
Iterate – faster you try things, the more successful ideas you have
Build Infrastructure to enable fast iteration of experiments
14. Vector-based search:
Word is represented as vector.
Vector captures word meaning – its semantics.
Words with similar meanings get similar vectors.
In Bing, we trained a GloVe model over
Bing’s query logs and generated 300-
dimensional vector, enabled by deep
neural network.
Deep Learning: Semantic Understanding
15. Vector-based Retrieval
DL-generated vectors semantically represent queries, documents and passages
Doc retrieval based on query-doc-passage semantic similarity (vector distance)
AND
long
does
…
canned
how
Query: {how long
does a canned
soda last}
canned
does
how
long
…
Posting 1
Posting 2
Posting 3
Posting 4
…
Matching
BM25
Perfect Match
Sem. Similarity
Vector
Recall
…
(0.78, 0.8, 0.4, 0.3, …)
(0.75, 0.6, 0.1, 0.8, …)
…
Approximate Nearest Neighbor
Search (ANN)
RANKING STACK
16.
17. Bing AI Platform – Bing QnA
DATA PREP BUILD TRAIN DEPLOY INTELLIGENT APPS
ACTIONINTELLIGENCEDATA
Stored on
Cosmos
Æther with
TensorFlow
Philly
Ultra-fast Inferencing
using FPGAs
21. AI Transforms all businesses across Microsoft
AI is now done in almost every team
Products that don’t seem to obviously use AI are powered by it
Using the same tools and infrastructure to accelerate new teams
22. Took some time to warm up to the ideas…
To: Eric Boyd
From: xxxxx
Subject: ABC Experiment
…we have decided to ship the
XXX feature without running
the experiment first. We’re
quite confident in the
feature.
Our teams have a rich history
of shipping features without
any experimentation, something
that may be alien for folks
from Bing.
To: xxxxx
From: Eric Boyd
Subject: Re: ABC Experiment
I have a rich history of
driving my car with my eyes
closed. I’m quite confident I
know the way to work and have
only crashed a few times.
23.
24.
25. 0
5k
10k
15k
Dailykeptcount
Rule-based Machine learning
Models trained on
exploration data
Filter using user
interactions
Offline model
Machine Learning for SmartArt Suggestions
0
5
10
15
20
25
30
35
40
45
1-Apr 15-Apr 29-Apr 13-May 27-May 10-Jun 24-Jun 8-Jul 22-Jul 5-Aug 19-Aug 2-Sep 16-Sep 30-Sep 14-Oct 28-Oct 11-Nov 25-Nov
Slidekeptrate
26. Machine Learning for SmartArt Suggestions
0
50k
100k
150k
Dailykeptcount
Rule-based Machine learning
0
5
10
15
20
25
30
35
40
45
1-Apr 15-Apr 29-Apr 13-May 27-May 10-Jun 24-Jun 8-Jul 22-Jul 5-Aug 19-Aug 2-Sep 16-Sep 30-Sep 14-Oct 28-Oct 11-Nov 25-Nov
Slidekeptrate
More SmartArts,
trained on all data
Offline model
Filter using user
interactions
Models trained on
exploration data
27. Office – PowerPoint Designer
DATA PREP BUILD & TRAIN DEPLOY INTELLIGENT APPS
ACTIONINTELLIGENCEDATA
Stored on
Cosmos
Æther with
TLC (ML.NET)
Power Point
User Behavior
Data prep using
Azure Databricks
osi
31. Every business at Microsoft investing in AI
+ more!
1000s of Data Scientists & AI Developers
Classical ML & Deep Learning
Compliant & non-compliant data
Internal & OSS frameworks
Internal & OSS tools & languages
Deployments to Azure, AP, Windows, Mobile, Edge
32. Building your own AI models for Transforming Data into Intelligence
Prepare Data Build & Train Deploy
33. 8 Exabytes
30M events/sec
From 1B+ devices
Cosmos
Step 1: Prepare Data
GDPR compliant cloud data processing
system for near real-time ingestion &
processing
34. System for managing ML pipelines
Optimized for rapid prototyping, data reuse,
and collaboration
>20M Pipelines
>1.75M active Datasources (>2.7 total)
Æther
Step 2: Build and Train
36. Hyperparameter Tuning
As the models get more complex, the
space of hyperparameters to explore
increases exponentially!
Grid or Random Search becomes too
costly
Bayesian Optimization is used to optimize
Acquisition Functions
37. AI Platform
investments
Built an exabyte-scale data lake for everyone
to put their data of all types (structured and
unstructured)
Built AI tools approachable by any developer
Built machine learning tools for collaborating
across large experiment models
Summary
40. How do we get from raw point clouds to cloud intelligence?
Wall
Floor
Ceiling
Table
Chair
Background
Project Kinect for Azure
41. Depth and Deep Learning
Using the power of our AI tools and infrastructure, we can take that raw output and train a
model capable of high fidelity environment perception.
Input: Project
Kinect for Azure
Raw Depth +
Active Brightness data Labelling Tool
Labelled
“Ground Truth”
Training Set Test Set
Philly
CNTK model
Analytics Client
ONNX model
43. Labelled “Ground
Truth”
Raw Depth + Active
Brightness data
Labelling Tool Labelled “Ground
Truth”
Building the Training Data – How do we scale?
TABLE
50. ONNX enables models to be trained in one framework and transferred to another for inference.
CPUGPU
ML HW
DSPFPGA
High level API &
Framework Frontends
Hardware Vendor
Libraries & Devices
Any tools exporting ONNX models can benefit ONNX-compatible runtimes and libraries designed
to maximize performance on some of the best hardware in the industry.
Seamless Interoperability
ONNX.ai
52. Contribute
Get Involved
github.com/onnx
ONNX is a community project. We
encourage you to join the effort
and contribute feedback, ideas,
and code. Join us on Github.
Use ONNX
ONNX.ai
Start experimenting today. Check
out our Getting Started Guide,
Supported Tools, and Tutorials.
Follow Us
Stay up to date with the latest
ONNX news.
onnxai
onnxai
56. Intelligent Disease Prediction
Data Prep Build Train Deploy Intelligent Apps
Azure Machine
Learning
IoT Hub
WindowsML
IOT Edge
Model:
DenseNet-121
Code:
Keras +
TensorFlow
National Institute
of Health
Chest Xray Data
112,120 images
14 pathology labels
30,805 unique patients
Visual Studio
Tools for AI
Azure
Deep Learning GPU VM
VSTS +
CI/CD
CosmosDB +
Azure Functions
NuGet
59. With , you don’t have to be the
size of Bing to solve the problem
60. Bringing the best of AI to Azure and the best of Azure to AI
Pre-Build AI
Azure Cognitive Services
Conversational AI
Azure Bot Services
Custom AI
Azure Machine Learning
61. Integrated with Azure Machine Learning
Create new deep learning projects easily
Scale Out with Azure Batch AI
Monitor model training progress & GPU utilization
Visualize your model processing with integrated open
tools like TensorBoard
Get started quickly with the Samples Gallery
Productive AI developer tools to train
models and infuse AI into your apps
62. The Machine Learning framework made by and for .NET developers
Proven & Extensible
Open Source
Supported on Windows, Linux, and macOS
Developer Focused
Join at github.com/dotnet/machinelearning
Customizable
ML.NET Preview
Cross-platform open Source Machine learning framework for .NET
Extensively used across Microsoft: Windows, Bing, Azure
High productivity for complete workflow
Extensible to other frameworks (TensorFlow, CNTK…)
63. Microsoft scale Tools/Services available/coming to you
Cognitive Toolkit (CNTK)
ML.Net
ONNX
Azure Batch AI
Hyper Parameter Tuning in Azure ML
Project BrainWave (FPGA model acceleration) in Azure ML
Visual Studio Tools for AI
And more coming soon…