Ubiquitous AI accelerators will power the explosion of AI from in-device uses to the edge, IoT and the cloud

1. 1
AI Everywhere:
Democratizing
Artificial Intelligence
with an open platform
and end-to-end ecosystem
Michael Gschwind, PhD
Chief Architect, AI and Intelligent Systems
Silicon Valley Computing Lab

2. 2
Artificial Intelligence
Specific, human-like
tasks
Classify and understand
real world data: recognize
objects, understand
language, interpret data
Machine-learning
techniques
Automate model build to
iteratively learn from data
and find hidden insights
without programmer
Self-learning
After initial training and
deployment, continue to
learn on its own based on
data it receives
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

3. 3
3
Big
Data
Artificial
Intelligence &
Intelligent
Applications
Machine
Learning
Deep
Learning
(Neural Nets)
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

4. 4
The Time is Right
Sufficient computing
capacity for good models
SuperVision (Alexnet)
wins ILSVRC 2012 image
recognition competition
Finally more neurons than a
fruit fly! Deep Neural
Networks and many FLOPS!
Deep Learning as a way to
“automate” feature extraction!
“traditional CV”
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

5. 5
Deep Learning
Deep Learning
ImageNet: 5x improvement with Artificial Neural Networks (ANN) in 6 years
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

6. 6
Machine and Deep Learning
What we do “without even thinking about it”…..we recognize a bicycle
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

7. 7
Neural Networks: “Inspired” by Biology
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

8. 8
Source: blog.openai.com/ai-and-compute/
AlexNet to AlphaGo Zero:
2x Performance Growth every 3.5 months
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

9. 9
Key Factors Supporting AI Improvements
Algorithmic
innovation
Neural network
architecture innovation;
and improved algorithms
for training
Training
data
Improved collection,
labeling and preprocessing
of increasingly larger
training corpora
Compute
capacity
Steadily increasing compute
capacity enables training of
more sophisticated
networks with more data
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

10. 10
Beyond Moore’s Law
Moore’s
Law
Limitations of device physics
prevents continued
exponential performance gains
from chip manufacturing
Architectural
Innovation
Improved efficiency of
application-optimized
accelerators improves
performance and efficiency
Hardware/Software
Ecosystem Optimization
Complimentary optimizations
of the AI ecosystem improve
application performance and
developer productivity
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

11. 11
11
Gen 1:
Acceleration
PCIe-attached Accelerator
Gen 2:
Multi-Accelerator
Multi- and Many-
Accelerator Systems
Gen 3:
Clustered Accelerators
Clustered Many-Accelerator
Systems with Optimized
Communication Fabric
Three Generations of Accelerated AI Systems
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

12. 12
The Real World: Big Data!
12
Source: Augmenting Pathology Labs with
Big Data and Machine Learning
(The Next Platform)
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

13. 13
AI Everywhere
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

14. 14
for you
by you
AI Everywhere
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

15. 15
for you
by you
End to End Interoperability
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

16. 16
Autonomous Intelligence
Sustainable
operation
Reduce power consumption
and network congestion
Disconnected
operation
Fail-safe independent
operation without network
Real-time
response
Respond to local conditions
without network delay
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

17. 17
Deploying DL Models
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

18. 18
First AI Accelerator in Mobile:
Kirin 970 with NPU
ML acceleration API and library for NPU
API implemented as ML Mobile OS service
Mobile OS assigns different tasks to purpose-
optimized processing unit: NPU, GPU, CPU, etc.
AI Applications
AI Framework
Library
NPU (Neural Processing Unit) Ecosystem
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

19. 19
In-Device AI Acceleration
• Incubate rich ecosystem with independent
developers
• Foster innovation to enable new AI use cases
• Create end user choice through application
diversity
AI Applications
AI Framework
Library
Create rich user-centric AI software ecosystem
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

20. 20
“Affordable AI”
• Integrated in all devices to enable AI
exploitation
• Deliver AI performance for all users
• Efficient resource usage for mobile and edge
application
AI Applications
AI Framework
Library
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

21. 21
Industry Initiative: MLPerf
Industry initiative to create
standardized cross-platform,
cross-vendor performance
benchmark for Machine
Learning
Benchmark Suite
Set of AI Benchmarks for
AI Software Frameworks
and Hardware Systems
Model Build and Use
Benchmarks for Model Build
(Training) and Model Use
(Inference), in the Cloud
(Data Center) and at the
Edge (Mobile, IoT).
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

22. 22
AI Model Patterns: In-Device Model
In-Device
AI model use
In-Device model use pattern
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

23. 23
AI Model Patterns: In-Device Model
In-Device
AI model use
Data upload
In-Device model use pattern
Model update
Cluster training
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem
Data storage

24. 24
AI Model Patterns: Cloud Model
Cloud
AI model use
Data upload
Cloud AI model use pattern
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

25. 25
AI Model Patterns: Cloud Model
Cloud
AI model use
Data upload
Cluster training
Model update
Cloud AI model use pattern
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem
Data storage

26. 26
AI Model Patterns: Mobile Use Cases
Combine In-Device and Cloud AI Model
In-Device
AI model use
Data upload
Hybrid In-Device/Cloud
AI model use pattern
Cloud
AI model use
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

27. 27
AI Model Patterns: Mobile Use Cases
Combine In-Device and Cloud AI Model
In-Device
AI model use
Data upload
Data storage
Hybrid In-Device/Cloud
AI model use pattern
Cloud
AI model use
Model update
Cluster training
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

28. 28
Information Technologies Defining Their Era
PC Era
Process and store data
Internet Era
Find and share data
AI Era
Understand data and
discover information
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

29. 29
Many More Breakthroughs To Come
AlphaGo Zero
AI trains itself using only Go
rules: faster training than
from past (human) games
AlphaGo Teach
AI coaches human players
to become better by using
AI-discovered knowledge
AlphaGo
AI learns from past games:
long knowledge transfer
(training) from past games
0
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

30. 30
“AI is whatever hasn't been done yet”
“Every time we figure out a piece of it, it stops being
magical; we say, 'Oh, that's just a computation.’ ”
(Rodney Brooks)
30
(Tesler’s Theorem)
Michael Gschwind, AI Everywhere: Democratizing Artificial
Intelligence with an open platform and end-to-end ecosystem

31. 31
AI Everywhere:
democratize AI with
an open platform
and end-to-end
ecosystem
user-focused
open
resilient
sustainable
from device to cloud