Putting machine learning on edge nodes adds local control, reduces privacy concerns, and doesn't require a fast internet connection. uTensor and CMSIS-NN bring ML to the cheapest of computers: microcontrollers. $2 cost and 1 cm2 in size they're found everywhere.

1. Machine learning on
one one square centimeter
Jan Jongboom, Arm
Daho.am
24 July 2018

2. Jan Jongboom (LinkedIn, GH, Twitter)
jan.jongboom@arm.com

5. http://www-03.ibm.com/ibm/history/ibm100/us/en/icons/ibm700series/impacts/

8. https://cdn2.i-scmp.com/sites/default/files/styles/980x551/public/images/methode/2017/05/23/6660b96e-3f9d-11e7-8c27-b06d81bc1bba_1280x720_183924.JPG?itok=ZmONr2a_

10. © 2018 Arm Limited 9
Machine learning

11. © 2018 Arm Limited 10
Downsides
I'm not rich enough to develop 5,000 custom
processors
Centralized
Lots of power required to send data back and forth
Ethical aspect
Not me

12. © 2018 Arm Limited 11

13. © 2018 Arm Limited 12
Microcontrollers
Small (1cm²)
Cheap (~1$)
Efficient (years on ba=ery)
Slow (max. 100 MHz)
Limited memory (max. 256K RAM)
Downsides
8 cm

14. © 2018 Arm Limited 13
Reinforcement learning
X
X
O
O
Content of lucifer box
Every box
resembles a state
of the board

15. © 2018 Arm Limited 14
Reinforcement learning
X
X
O
O
Content of lucifer box

16. © 2018 Arm Limited 14
Reinforcement learning
X
X
O
O
Content of lucifer box

17. © 2018 Arm Limited
X
14
Reinforcement learning
X
X
O
O
Content of lucifer box

18. © 2018 Arm Limited
X
14
Reinforcement learning
X
X
O
O
Content of lucifer box
Rules
Lose: Remove bead
Draw: Place 1 bead back
Win: Place 3 beads back

19. © 2018 Arm Limited
X
15
Reinforcement learning
X
X
O
O
Content of lucifer box
O

20. © 2018 Arm Limited 16
Reinforcement learning
X
X
O
O
Content of lucifer box

21. © 2018 Arm Limited 16
Reinforcement learning
X
X
O
O
Content of lucifer box

22. © 2018 Arm Limited
X
16
Reinforcement learning
X
X
O
O
Content of lucifer box

23. © 2018 Arm Limited 17
Reinforcement learning
X
X
O
O
Content of lucifer box

24. © 2018 Arm Limited 18
Training vs. classification
Hundreds of different states
Need to encounter states many times
Training takes long!
Classification is however simple
Play the game, and you have to open up max. 4 drawers!

25. © 2018 Arm Limited 19
uTensor
Machine learning for microcontrollers
Runs in <256K RAM
TensorFlow compatible
Just does classification
Open source, Apache 2.0 license
Built on top of Mbed OS 5

26. © 2018 Arm Limited 20
Enabling new use cases
Sensor fusion
http://www.gierad.com/projects/supersensor/

27. © 2018 Arm Limited 21

28. © 2018 Arm Limited 22
Federated learning
https://research.googleblog.com/2017/04/federated-learning-collaborative.html
Enabling new use cases

29. © 2018 Arm Limited 23
LPWANs
http://ufldl.stanford.edu/tutorial/unsupervised/Autoencoders/
Enabling new use cases

30. © 2018 Arm Limited 24
Offline self-contained systems
https://os.mbed.com/blog/entry/streaming-data-cows-dsa2017/
Enabling new use cases

31. © 2018 Arm Limited 25

32. © 2018 Arm Limited 26

33. © 2018 Arm Limited 27
What do we see here?
MNIST data set
Training set: 60,000 images
Every drawing is downsampled to 28x28 pixels
Supervised learning through backpropagation
https://github.com/uTensor/tf-node-viewer/blob/master/deep_mlp.py

34. © 2018 Arm Limited 28
Multi-layer perceptron (MLP) classification
28x28 = 784
Neuron
Output
9
Matrix multiplication (weight), bias, then activation function
Potential outputs

35. © 2018 Arm Limited 29
Memory usage
Matrix multiplication in first hidden layer dominates RAM usage:
Input elements: 784
Number of neurons (1st layer): 128
Number of weight (input to 1st layer): 128 * 784
Resulting values (Pre-activation function): 128
Data type: 8-bit integer (1 byte)
1 byte * (784 + (128 * 784) + 128) = 98.891 kB

36. © 2018 Arm Limited 30
CMSIS-NN
New neural network kernel functions
Leverages the DSP/SIMD functions in silicon
See speedup of 4-5x
Hardware acceleration for convolution, pooling, etc.
uTensor will be built on top of CMSIS-NN

38. https://github.com/uTensor/ADL_demo

39. © 2018 Arm Limited 33
Recap
1. Machine learning is cool
2. Machine learning on the edge is even cooler
3. Buy a dev board and clone uTensor (https://github.com/uTensor/uTensor)
4. ???
5. PROFIT!!!

40. © 2018 Arm Limited 34
https://labs.mbed.com
Slides: http://janjongboom.com
Thank you!