Successfully reported this slideshow.
Your SlideShare is downloading. ×

Tiny intelligent computers and sensors - Open Hardware Event 2020

More Related Content

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Tiny intelligent computers and sensors - Open Hardware Event 2020

  1. 1. Jan Jongboom Open Hardware Event 1 April 2020 Tiny intelligent computers and sensors
  2. 2. Jan Jongboom CTO and co-founder, Edge Impulse jan@edgeimpulse.com
  3. 3. 3 Typical industrial sensor in 2019 Vibration sensor (up to 1,000 times per second) Temperature sensor Water & explosion proof Can send data >10km using 25 mW power Processor capable of running >20 million instructions per second
  4. 4. 4 But... what does it actually do? Once an hour: • Average motion (RMS) • Peak motion • Current temperature
  5. 5. 5 99% of sensor data is discarded due to cost, bandwidth or power constraints. https://www.mckinsey.com/~/media/McKinsey/Business%20Functions/McKinsey%20Digital/Our%20Insights/ The%20Internet%20of%20Things%20The%20value%20of%20digitizing%20the%20physical%20world/The- Internet-of-things-Mapping-the-value-beyond-the-hype.ashx
  6. 6. 6 Lots of interesting events get lost Peak
  7. 7. 7 Single numbers can be misleading updown circle avg. RMS 3.3650 3.3515
  8. 8. 8 On-device intelligence is the only solution 🚫
  9. 9. 9 On-device intelligence is the only solution Vibra&on pa+ern heard that lead to fault state in a weekTemperature varies in a way that I've never seen before Machine oscillates different than all other machines in the factory
  10. 10. 10 On-device intelligence is the only solution Temperature varies in a way that I've never seen before (0x1)
  11. 11. 11 Drawing conclusions directly on the sensor will drastically increase usefulness (and allow us to move to higher value use cases)
  12. 12. 12 Interesting questions... Classification What's happening right now? Anomaly detection Is this behavior out of the ordinary? Forecasting What will happen in the future?
  13. 13. https://cdn2.i-scmp.com/sites/default/files/styles/980x551/public/images/methode/2017/05/23/6660b96e-3f9d-11e7-8c27-b06d81bc1bba_1280x720_183924.JPG?itok=ZmONr2a_
  14. 14. 15 ML is everywhere Customer segmentation Finding fraudulent transactions Recommendation systems Virtual assistants (Siri, Google Home) Spam classification
  15. 15. 16 Machine learning
  16. 16. 17 Downsides I'm not rich enough to develop 5,000 custom processors Centralized Costs lots of power and bandwidth Privacy Not me
  17. 17. 18 Learning tic-tac-toe X X O O Content of lucifer box Every box resembles a state of the board
  18. 18. 19 Learning tic-tac-toe X X X O O Content of lucifer box Rules Lose: Remove marble Draw: Place 1 marble back Win: Place 3 marbles back
  19. 19. 20 X X X O O O Learning tic-tac-toe Content of lucifer box You lose
  20. 20. 21 Learning tic-tac-toe X X X O O Content of lucifer box You win
  21. 21. 22 Learning tic-tac-toe X X O O Content of lucifer box
  22. 22. 23 Training vs. classification Hundreds of different states Need to encounter states many times Training takes long! Classification is however simple Play the game, and you have to open up max. 4 drawers!
  23. 23. 24 Machine learning on the edge Typically only inferencing, no training Typically more efficient than sending data over the network Signal processing is still key
  24. 24. 25 Enabling new use cases Sensor fusion http://www.gierad.com/projects/supersensor/
  25. 25. 26 Anomaly detection Enabling new use cases
  26. 26. 27 Enabling new use cases Livestock monitoring https://os.mbed.com/blog/entry/streaming-data-cows-dsa2017/
  27. 27. Machine learning is great at finding patterns in messy data (anything you can't reason about in Excel)
  28. 28. 29 In the industry Large push from Google, Arm in creating better software ecosystem (TensorFlow Lite, uTensor) Lots of new chips coming out (ETA Compute, Arm Cortex-M55) with hardware acceleration Making smaller neural networks: quantization, pruning, lottery tickets But... collecting and organizing high-quality data is hard!
  29. 29. 30 Signal processing is still key
  30. 30. 31 Edge Impulse - TinyML as a service Embedded or edge compute deployment options Test Edge Device Impulse Dataset Acquire valuable training data securely Test impulse with real-time device data flows Enrich data and generate ML process Real sensors in real time Open source SDK Free for developers: edgeimpulse.com
  31. 31. 32 v In practice https://www.flickr.com/photos/120586634@N05/14491303478
  32. 32. 33 Sheep activity tracker h+ps://pixabay.com/photos/sheep-curious-look-farm-animal-1822137/
  33. 33. 34 Capturing raw data 11 minutes of raw data over 4 classes. Collected on device, synced over WiFi
  34. 34. 35 Extracting features
  35. 35. 36 Training two models Neural network classifier Anomaly detec7on
  36. 36. 37 From model to device Signal processing, neural network and anomaly detec&on
  37. 37. 38 Conclusions back to cloud ♻ Sample for four seconds Classify Result differs? Message. Sheep is walking h+ps://pixabay.com/photos/sheep-curious-look-farm-animal-1822137/
  38. 38. 39 Device 80MHz Cortex-M4F processor 128 KiB RAM Time to analyze 1 second of ac&vity data (DSP + classifica&on + anomaly): 0.008 seconds
  39. 39. Demo 🚀
  40. 40. 41 How to get started? edgeimpulse.com tinymlbook.com
  41. 41. Recap The ML hype is real ML + sensors = perfect fit Start using the remaining 99% of sensor data edgeimpulse.com
  42. 42. 43 Thank you! Slides: janjongboom.com

×