The document discusses machine learning on microcontrollers. It describes how machine learning can now be run on small, cheap microcontrollers using techniques like reinforcement learning and frameworks like uTensor. This opens up new applications in areas like sensor fusion, federated learning, offline self-contained systems, and more. The document provides examples of models like MNIST digit classification that can run within the limited memory of microcontrollers.

1. Machine learning
on microcontrollers
Jan Jongboom
Tech Power Summit
14 April 2018https://gfycat.com/CreepyExcitableBird

5. http://www-03.ibm.com/ibm/history/ibm100/us/en/icons/ibm700series/impacts/

8. https://cdn2.i-scmp.com/sites/default/files/styles/980x551/public/images/methode/2017/05/23/6660b96e-3f9d-11e7-8c27-b06d81bc1bba_1280x720_183924.JPG?itok=ZmONr2a_

9. © 2018 Arm Limited 8
Machine learning

10. © 2018 Arm Limited 9
Downsides
I'm not rich enough to develop 5,000 custom
processors
Centralized
Lots of power required to send data back and forth
Ethical aspect
Not me

11. © 2018 Arm Limited 10
Microcontrollers
Small (1cm²)
Cheap (~1$)
Efficient (standby: 0.3 μA)
Slow (max. 100 MHz)
Limited memory (max. 256K RAM)
Downsides
2.7 inch

12. © 2018 Arm Limited 12
Reinforcement learning
X
X
O
O
Content of lucifer box

13. © 2018 Arm Limited 12
Reinforcement learning
X
X
O
O
Content of lucifer box

14. © 2018 Arm Limited
X
12
Reinforcement learning
X
X
O
O
Content of lucifer box

15. © 2018 Arm Limited
X
12
Reinforcement learning
X
X
O
O
Content of lucifer box
Rules
Lose: Remove bead
Draw: Place 1 bead back
Win: Place 3 beads back

16. © 2018 Arm Limited
X
13
Reinforcement learning
X
X
O
O
Content of lucifer box
O

17. © 2018 Arm Limited 14
Reinforcement learning
X
X
O
O
Content of lucifer box

18. © 2018 Arm Limited
X
14
Reinforcement learning
X
X
O
O
Content of lucifer box

19. © 2018 Arm Limited 15
Reinforcement learning
X
X
O
O
Content of lucifer box

20. © 2018 Arm Limited 16
Training vs. classification
Hundreds of different states
Need to encounter states many times
Training takes long!
Classification is however simple
Play the game, and you have to open up max. 4 drawers!

21. © 2018 Arm Limited 17
uTensor
Machine learning for microcontrollers
Runs in <256K RAM
TensorFlow compatible
Just does classification
Open source, Apache 2.0 license

22. © 2018 Arm Limited 18
Enabling new use cases
Sensor fusion
http://www.gierad.com/projects/supersensor/

23. © 2018 Arm Limited 19
Federated learning
https://research.googleblog.com/2017/04/federated-learning-collaborative.html
Enabling new use cases

24. © 2018 Arm Limited 20
LPWANs
http://ufldl.stanford.edu/tutorial/unsupervised/Autoencoders/
Enabling new use cases

25. © 2018 Arm Limited 21
Offline self-contained systems
https://os.mbed.com/blog/entry/streaming-data-cows-dsa2017/
Enabling new use cases

26. © 2018 Arm Limited 22

27. © 2018 Arm Limited 23
What do we see here?
MNIST data set
Training set: 60,000 images
Every drawing is downsampled to 28x28 pixels
Supervised learning through backpropagation
https://github.com/uTensor/tf-node-viewer/blob/master/deep_mlp.py

28. © 2018 Arm Limited 24
Multi-layer perceptron (MLP) classification
28x28 = 784
Neuron
Output
9
Matrix multiplication (weight), bias, then activation function
Potential outputs

29. © 2018 Arm Limited 25
Memory usage
Matrix multiplication in first hidden layer dominates RAM usage:
Input elements: 784
Number of neurons (1st layer): 128
Number of weight (input to 1st layer): 128 * 784
Resulting values (Pre-activation function): 128
Data type: 8-bit integer (1 byte)
1 byte * (784 + (128 * 784) + 128) = 98.891 kB

30. © 2018 Arm Limited 26
CMSIS-NN
New neural network kernel functions
Leverages the DSP/SIMD functions in silicon
See speedup of 4-5x
Hardware acceleration for convolution, pooling, etc.
uTensor will be built on top of CMSIS-NN

32. © 2018 Arm Limited 28
Speed
Measured on Cortex-M7 216 MHz, 133 KB memory used
3 convolution layers, 1 fully connected layer, 32x32 color images

33. © 2018 Arm Limited 29
Recap
1. Machine learning is cool
2. Machine learning on the edge is even cooler
3. Buy a dev board and clone uTensor (https://github.com/uTensor/uTensor)
4. ???
5. PROFIT!!!

34. © 2018 Arm Limited 30
https://labs.mbed.com
Jan Jongboom, Arm
Thank you!