1. Azure AI provides updates on advances in AI capabilities such as object recognition reaching human parity in 2016 and machine translation reaching human parity in 2018. 2. Responsible AI practices at Microsoft include interpretability, fairness, and privacy tools to ensure AI systems are understandable, unbiased, and protect user data. 3. Differential privacy and homomorphic encryption techniques allow training models and performing inferences on encrypted user data to enable private and confidential machine learning.

1. Azure AI –
Build 2020 Updates
SATO Naoki (Neo)
Senior Software Engineer
Microsoft

3. Fueled by breakthrough research
2016
Object recognition
human parity
2017
Speech recognition
human parity
2018
Reading comprehension
human parity
2018
Machine translation
human parity
2018
Speech synthesis
near-human parity
2019
General Language
Understanding human parity
2020
Document summary at
human parity

4. 1.8M
Hours of meetings
transcribed in real-time
1B
PowerPoint Designer
slides used
Tested at scale in Microsoft solutions
80M
Personalized experiences
delivered daily
Machine translation
human parity
Object detection
human parity
Switchboard
Switchboard cellular
Meeting
speech
IBM Switchboard
Broadcast speech
Speech recognition
human parity
Conversational Q&A
human parity
First FPGA deployed
in a datacenter

5. Power Platform
Power BI Power Apps Power Automate Power Virtual Agents
Azure
58 Regions 90+ Compliance Offerings $1B Security investment per year95% Fortune 500 use Azure
Azure AI
ML platform
Customizable models
Vision, Speech, Language, Decision
Scenario-specific services
Cognitive Services
Azure Machine Learning
Data Platform App Dev Platform & tools Compute
Cognitive SearchBot Service Form Recognizer Video Indexer

6. Cloud
CPU, GPU, FPGA
Datasets
Profiling, Drift, Labeling
Inferencing
Batch, Realtime
MLOps
Reproducible, Automatable, GitHub, CLI, REST
Experience
SDK, Notebooks, Drag-n-drop, Wizard
Edge
CPU, GPU, NPU
IoT Edge
Security, Mgmt, Deployment
Compute
Jobs, Clusters, Instances
Model Registry
Models, Images
Training
Experiments, Runs

7. Responsible AI

8. PracticesPrinciples Tools
AETHER committee
The Partnership on AI
Guidelines for
Human-AI Design
Homomorphic Encryption
Interpret ML
Differential Privacy
Data Drift
Secure MPC
Guidelines for
Conversational AI
Fairness
Reliability
Privacy
Inclusivity
Accountability
Transparency

9. Why responsible AI?
Report

10. Understand
Interpretability
Fairness
Azure Machine Learning
Responsible ML
Control
Audit trail
Datasheets
Protect
Differential privacy
Confidential machine learning

11. Understand
Interpretability
Fairness
Azure Machine Learning
Responsible ML
Control
Audit trail
Datasheets
Protect
Differential privacy
Confidential machine learning

12. Loan Application Decisions
Azure Machine Learning
How does it decide who
to accept or reject?
Is my model fair?
Create a model for loan
application acceptance

13. Loan Application Decisions
Azure Machine Learning
How does it decide who
to accept or reject?
Is my model fair?
Create a model for loan
application acceptance

14. Fairness in AI
There are many ways that an AI system can behave unfairly
A voice recognition system
might fail to work as well for
women as it does for men
A model for screening loan or job
application might be much better at
picking good candidates among white
men than among other groups
Avoiding negative outcomes of AI systems for different groups of people

15. Assessing unfairness in your model
https://github.com/fairlearn/fairlearn
Fairness
assessment:
Usecommonfairness metrics
andaninteractive dashboardto
assess which groups of peoplemay
benegatively impacted
Model formats:
Python models using scikit predict
convention, Scikit, Tensorflow,
Pytorch, Keras
Metrics:
15+ common group fairness metrics
Model types:
Classification, Regression
Fairness mitigation:
Use state-of-the-art algorithms
to mitigate unfairness in your
classificationandregressionmodels

16. Fairness Assessment
Input Selections
Sensitive attribute
Performance metric
Assessment Results
Disparity in performance
Disparity in predictions

17. Fairness Mitigation
Fairness Criteria
Demographic Parity
Equalized Odds
Mitigation Algorithm
Postprocessing Method
Reduction Method

18. Loan Application Decisions
Azure Machine Learning
How does it decide who
to accept or reject?
Is my model fair?
Create a model for loan
application acceptance

19. Understand and debug your model
Interpret
Glassbox and blackbox
interpretability methods
for tabular data
Interpret-
community
Additional interpretability
techniques for tabular data
Interpret-text
Interpretability
methods for text data
DiCE
Diverse Counterfactual
Explanations
Blackbox models:
Model formats:
Python models using scickit predict
convention, Scikit, Tensorflow, Pytorch, Keras
Explainers:
SHAP, LIME, Global Surrogate,
Feature Permutation
Glassbox Models:
Model types:
Linear Models, Decision Trees, Decision Rules,
Explainable Boosting Machines
AzurML-interpret
AzureML SDK wrapper for Interpret
and Interpret-community
https://github.com/interpretml

20. Interpretability
approaches
Glassbox
models
Blackbox
explanations

21. Fever?
Internal
bleeding?Stay
home
Stay
home
Go to
hospital
Models designed
to be interpretable.
Lossless
explainability.
Glassbox
models
Decision trees
Rule lists
Linear models
Explainable
Boosting Machines

22. Explain any
ML system.
Approximate
explainability.
Blackbox
explanations
Model
Explanation
Perturb
inputs
Analyze
Shap
Lime
Partial dependence
Sensitivity analysis

23. Responsible ML resources
Microsoft Responsible AI
Resource Center
https://aka.ms/RAIresources
Azure Machine Learning
https://azure.microsoft.com/en-us/services/machine-
learning/
https://docs.microsoft.com/en-us/azure/machine-
learning/concept-responsible-ml
Responsible Innovation Toolkit
https://docs.microsoft.com/azure/architecture/guide/resp
onsible-innovation
FairLearn
https://github.com/fairlearn
https://aka.ms//FairLearnWhitepaper
https://docs.microsoft.com/azure/machine-
learning/concept-fairness-ml
InterpretML
https://github.com/interpretml
https://aka.ms//InterpretMLWhitepaper
https://docs.microsoft.com/azure/machine-learning/how-
to-machine-learning-interpretability

24. Understand
Interpretability
Fairness
Azure Machine Learning
Responsible ML
Control
Audit trail
Datasheets
Protect
Differential privacy
Confidential machine learning

25. Differential Privacy for Machine Learning and Analytics
https://github.com/opendifferentialprivacy
Native Runtime
C,C++, Python, R
Validator
Automatically stress test DP
algorithms
Data Source Connectivity
Data Lakes, SQL Server, Postgres,
Apache Spark, Apache Presto and
CSV files
Privacy Budget
Control queries by users

26. WhiteNoise
Privacy Module
Report
Budget Store
BUDGET
User Private
Dataset
Submits a query
Receives a
differentially
private report
Mechanism adds
noise
Private data
Dataset checks
budget and access
credentials
Checks
budget and
private
compute
Credentials to
access the data
https://github.com/opendifferentialprivacy

27. Confidential ML Technologies
Privacy
Barrier
Privacy
Barrier
Confidential from Data
Scientist
Encrypted using Hardware

28. Homomorphic Encryption
Decrypt(Encrypt(A) + Encrypt(B)) = A + B
Decrypt(Encrypt(A) * Encrypt(B)) = A * B
Privacy
Barrier
Homomorphic encryption allows
certain computations to be done on
encrypted data, without requiring
any decryption in the process:
Different from classical encryption
like AES or RSA:

29. Microsoft SEAL
Open-source homomorphic encryption
library
Developed actively since 2015
Recently released v3.5
Available at GitHub.com/Microsoft/SEAL
Supports Windows, Linux, macOS, Android,
FreeBSD
Written in C++; includes .NET Standard
wrappers for public API
From open source community:
PyHeal (Python wrappers from Accenture)
node-seal (JavaScript wrappers)
nGraph HE Transformer (from Intel)

30. Private Prediction
on Encrypted Data
Through a trained machine
learning model, private prediction
enables inferencing on encrypted
data without revealing the content
of the data to anyone.
Microsoft SEAL can be deployed in
a variety of applications to protect
users personal and private data
Privacy Barrier
[cryptographic]
Medical prediction

31. Responsible ML Resources
 Microsoft Responsible AI Resource Center
 https://aka.ms/RAIresources
 Azure Machine Learning
 https://azure.microsoft.com/en-
us/services/machine-learning/
 https://docs.microsoft.com/en-
us/azure/machine-learning/concept-
responsible-ml
 OpenDP
 http://opendp.io/
 https://twitter.com/opendp_io
 WhiteNoise
 https://github.com/opendifferentialprivacy
 https://docs.microsoft.com/azure/machine-
learning/concept-differential-privacy
 https://docs.microsoft.com/azure/machine-
learning/how-to-differential-privacy
 https://aka.ms/WhiteNoiseWhitePaper
 SEAL
 https://github.com/Microsoft/SEAL
 https://docs.microsoft.com/azure/machine-
learning/how-to-homomorphic-encryption-
seal
 https://aka.ms/SEALonAML

32. AI at Scale

33. Next-gen AI capabilities
To accelerate the change we needed,
we took advantage of three trends
Transfer
learning
Large pre-trained self
supervised networks
Culture
shift

34. Microsoft Turing NLG
5 b
7.5 b
10 b
12.5 b
15 b
17.5 b
Spring ‘18 Summer ‘18 Autumn ‘18 Winter ‘19 Spring ‘19 Summer ‘19 Autumn ‘19 Winter ‘20
2.5 b
ELMo
94m
GPT
110m
BERT - large
340 m
Transformer
ELMo
465m
GPT-2
1.5b
MT-DNN
330m
XLNET
340m
XLM 665m
Grover-Mega
1.5b
RoBERTa
355m DistilBERT
66m
MegatronLM
8.3b
T-NLG
17b
Numberofparameters

35. Custom capabilities through reuse
Turing
NLP

36. Microsoft Bing
Intelligent Answers which summarize the web

37. Microsoft Word Smart Find
Search inside your document as easily as is to search the web

38. Microsoft SharePoint Inside Look
Find that document you’re looking for by browsing automatic document summaries

39. Microsoft Outlook Suggested Replies
Save time by responding with AI assisted suggestions

40. AI at Scale for everyone

41. Collaboration with OpenAI
Hosted in Azure
285,000 CPU cores, 10,000
GPUs, 400 Gbps for each
GPU server
Top 5 in Top 500 SCs
https://blogs.microsoft.co
m/ai/openai-azure-
supercomputer/

42. Learn more:
• AI at Scale introduction: aka.ms/AIatScale
• AI at Scale Deep Dive: aka.ms/AIS-DeepDive
• DeepSpeed library: github.com/microsoft/DeepSpeed
• ONNX Runtime: aka.ms/onnxruntime
• Try T-NLG in the Companion App: aka.ms/Build-AIS

43. Rethinking the AI Stack
NVidia GPUs
Intel FPGAs
NVLink
Infiniband
DeepSpeed allowing
for training models 15x
bigger, 10x faster on
the same infrastructure
ONNX Runtime
Central AI group
coordinating bringing
the best of research
into products
All available on Azure and GitHub for everyone!

44. © Copyright Microsoft Corporation. All rights reserved.