In this deck, Jem Davies (VP Engineering and ARM Fellow) gives a brief introduction to Machine Learning and explains how it is used in devices such as smartphones, autos, and drones. "I do think that machine learning altogether is probably going to be one of the biggest shifts in computing that we'll see in quite a few years. I'm reluctant to put a number on it like -- the biggest thing in 25 years or whatever," said Jem Davies in a recent investor call. "But this is going to be big. It is going to affect all of us. It affects quite a lot of ARM, in fact." Watch the video presentation: http://insidehpc.com/2017/03/slidecast-arm-steps-machine-learning/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

1. ©ARM 2016
ARM and Machine Learning
Jem Davies
ARM Fellow &VPTechnology
Media Processing Group, Imaging &Vision Group
December 12 2016

2. ©ARM 20162
 Jem Davies - ARM Fellow andVP ofTechnology - Media
Processing and Imaging &Vision Groups
 Working on multimedia, computer vision and machine learning
 13 years at ARM
 Mathematician turned chemist, turned software engineer…
 … turned architect
 … turned tech future predictor
 Glider pilot instructor, fireworks lighter
and scuba diver…
 … what next?
Who am I?

3. ©ARM 20163
Machine Learning overview
Speech recognitionRobotics Home security
 Machine Learning makes smart connections to previously
encountered concepts
 It’s useful when:
• We don’t have algorithms…
• but we do have a lot of data
Image recognition

4. ©ARM 20164
We use Machine Learning technology every day
Speech
recognition
Predictive text
Face tracking
camera
Fingerprint ID
Computational
photography
Digital
assistant

5. 5 ©ARM 2016
Definitions and
recent developments

6. ©ARM 20166
The AI landscape
AI
Machine
Learning
Computer
Vision
Natural
language
processing
Reasoning
Search and
Optimization
Constraints
and control
theory
Artificial
Intelligence
Machine
Learning
Deep
Learning
Algorithms:
CNNs,
RNNs, etc.

7. ©ARM 20167
Key terms and definitions
Artificial Intelligence
• The broadest term - applying to any technique enabling computers to mimic
human intelligence
Machine Learning
• A subset of AI including techniques enabling computers to improve at tasks
with experience. Includes deep learning
Deep Learning/Neural Networks
• A branch of machine learning that attempts to model real life ideas in data by
using a deep graph with multiple processing layers
Algorithms
• DNNs, CNNs, RNNs, SGEMM etc.
ComputerVision
• An interdisciplinary field that allows computers to gain understanding from
digital images or videos. Many computer vision applications use ML algorithms.

8. ©ARM 20168
Recent developments in deep learning
 The image detection benchmark ImageNet has
reached nearly 100% accuracy
 Largely due to Neural Networks
 Research groups feel it’s not useful to work on
ImageNet anymore (it’s a solved problem)
 GoogLeNet, Inception, etc.
 The successful approach used for ImageNet has
spread to other domains, yielding great
improvements

9. ©ARM 20169
Why did this happen ?
The Machine Learning learning loop

10. ©ARM 201610
Basic neural network
[Inputs] [Output]
w2
w1
w3
wn
wn-1
y
)(
1
zHy
xwz
n
i
ii
=
= ∑=
x1
x2
x3
xn-1
xn
Squashing function
Sigmoid: y=1/(1+e^(-z))

11. 11 ©ARM 2016
Machine Learning computation
in the cloud and on-device

12. ©ARM 201612
Dissecting the Machine Learning process
Generate inference engines
Distribute & optimize
Training engine
Inference engine
Platform acceleration libraries
ARM Cortex + Mali + Others
Heterogeneous tools
Cloud side
Device side

13. ©ARM 201613
Machine Learning process on ARM
Middleware
Cloud services / applications Cloud applications
Applications
CPU GPU
(Cache-coherent interconnect)
HW CoresCV/ML IP
HW Interconnect
HW
SW
Metadata streams, thumbnails etc.
Device
Cloud
Cloud service SDK
+ analytics
Cloud APIs
Training engine
Inference engine
Platform acceleration libraries
ARM Cortex + Mali + Others
Heterogeneous tools
Cloud side
Device side

14. 14 ©ARM 2016
It is easy to recognise speech
It is easy to wreck a nice beach

15. ©ARM 201615
Automatic speech recognition is not easy
 Active research area for over 50 years!
 Significant attention recently due to large amount of
cloud computing
 Neural Networks has improved performance
 Siri, Google Now, Cortana, Amazon Echo now creating
usable applications
 Applications
 Safety and security
 Interactions with smaller or hands free devices (e.g. wearables)
 Improved accessibility for hearing-impaired
 Allows indexing of spoken words

16. ©ARM 201616
ASR use cases and Machine Learning
 Large Vocabulary Continuous Speech Recognition (LVCSR)
 Dictation/transcription, virtual assistant
 Requires dictionary, knowledge of grammar
 Keyword spotting of simple commands
 “OK Google”, “Set alarm for 7”
 Sound monitoring
 Early/automatic anomaly detection

17. ©ARM 201617
Keyword spotting
 Listen for certain words/phrases
 Only need to learn certain words
 “Okay Google”
 “Play music”
 Simpler algorithm
 No knowledge of grammar
 Only needs acoustic model
 Algorithm must run locally on device
 Potentially always listening
 Power consumption vitally important
Speech Signal
Acoustic model
Features
Words Play music
Feature extraction

18. 18 ©ARM 2016
ComputerVision

19. ©ARM 201619
Computer vision pipeline
Pre-processing
Feature
extraction
Detection and
segmentation
High-level
processing
Image acquisition
Output
Application
ISP/GPU GPU/CPU/DSP/Spirit CV CPU

20. 20 ©ARM 2016
ARM and Machine Learning

21. ©ARM 201621
Machine Learning runs on ARM-powered client
devices today

22. ©ARM 201622
Caffe framework deep learning on ARM
Video hosted on youtube
https://www.youtube.com/watch?v=k4ovpelG9vs&t=13s

23. ©ARM 201623
Deep learning frameworks on ARM
 DNN used for real-time detection of objects
 Based on the Caffe deep learning framework
 Detection entirely on the mobile device – not
cloud based
 Optimized for ARM Cortex CPU or Mali GPU
 Running Machine Learning on the GPU frees
up the CPU for other tasks
 Optimised libraries also being developed for
TensorFlow and OpenVX
2.7 5.5
64% 20%

24. ©ARM 201624
Growth MaturityInnovation
Machine
Learning
Platforms
& Tools
Conclusions
 The rapid uptake in Machine Learning is
going mainstream, affecting compute
everywhere
 Machine Learning dramatically increases
compute demands
 As much as possible, Machine Learning
workloads should run locally on device,
not on remote servers
 Machine Learning is driving demand for advanced ARM
processors and accelerator IP
 Machine Learning is having a significant impact on ARM’s
roadmap for future processors and architectures

25. The trademarks featured in this presentation are registered and/or unregistered trademarks of ARM Limited
(or its subsidiaries) in the EU and/or elsewhere. All rights reserved. All other marks featured may be
trademarks of their respective owners.
Copyright © 2016 ARM Limited
©ARM 2016
Thank you for listening