Downloaded 72 times
Uploaded byAndri Yadi
AIoT: Intelligence on Microcontroller
The document discusses the integration of Artificial Intelligence (AI) with the Internet of Things (IoT), highlighting its architecture, benefits, and applications. It emphasizes moving AI capabilities to the edge for improved performance, lower latency, and better privacy, while also detailing the tools and hardware available for edge AI solutions. Additionally, it outlines practical use cases, including smart trash management and activity recognition, while promoting a community-supported platform for makers in Indonesia.
More Related Content
Similar to AIoT: Intelligence on Microcontroller
AIoT: Intelligence on Microcontroller
- 2. AIoT: Intelligence on Microcontroller Intro& Live Demo ULM | Banjarbaru - Aug 31, 2019 Andri Yadi CEO, DycodeX
- 3. Andri Yadi Co-founder &CEO of DycodeX Deputy Chairman, Indonesia IoT Association a (at) dycodex.com | http://andriyadi.com A Physicist, Developer, Maker, Community Guy, Entrepreneur About Me MicrosoM Most Valuable Professional (MVP) of Azure for 12 years Code for food & passion for 20 years Break & make electronic stuTs for 22 years Trying to change the world through entrepreneurship, 15 years now
- 4.
- 5. IoT Common (Simplified)Architecture Gateways / Base Station Rule + Alert, Internet CloudEdge Ingestion & Storage Things Visualization User Apps Devices & Firmware Mgt. Analytics (lots of them) Radio
- 6. What if weput AI into IoT?
- 8.
- 10. Ingestion / API Intelligence EngineInternet AI-powered, Cloud-backedApplication Edge Cloud Simplified Architecture Inference result Raw data
- 11. AI + IoT IoTis about automated data collecting, storing, visualisation, and reacting, in massive scale Combining AI + IoT enables a lot of use cases & business models! “Data Supplier” AI is about making sense of plethora of data, generating insights & recommendation, and predicting the future outcome “Data Miner”
- 12. Hyperlocal Weather Prediction Solarsensor Air temperature, humidity, barometric pressure Rain gauge Wind speedWind direction “Cheap” weather station on each blocks (city, farm, …) Air quality Weather Prediction Use Cases: Farming Online “Ojek” Fare and Distribution Logistics Flooding Damage predictionDeep Learning Model Examples: IBM Deep Thunder SmaRphone data
- 13. Ingestion / API Azure Cognitive Service:Face API Internet Example: Gender & Emotion Recognition Edge Cloud Microcontroller + Camera Sensor Inference result (JSON) Image
- 14. Ingestion / API Azure Cognitive Service:Face API Internet Example: Gender & Emotion Recognition Edge Cloud Microcontroller + Camera Sensor Inference result (JSON) Image
- 15. Ingestion / API Azure Cognitive Service:Face API Internet Example: Gender & Emotion Recognition Edge Cloud Male, Neutral Microcontroller + Camera Sensor Inference result (JSON) Image
- 16. Demo: Gender &Emotion Recognition in the Cloud hPps://www.youtube.com/watch?v=vvQfYqrUwFg
- 17. …and those arecloud-powered Artificial Intelligence Source: https://www.riskgroupllc.com/wp-content/uploads/Cloud-Powered-AI.jpg
- 18. AI on TheCloud Pros: You can always have more, just scale “Unlimited” computing power: for ML learning & inference “Unlimited” storage: for AI model and dataset Easier to maintain Easier to update (soMware, AI model/algorithm), to secure, etc Cons: High latency: as data need to transferred to cloud Must be online: to exchange data with cloud. Connectivity can be challenges for cecain use cases Potential privacy issue: your face goes to the cloud, who know what will happen
- 19. AI on TheCloud - Cons High latency Must be online Potential privacy issue What If… We move intelligence closer or at the Edge
- 20. Ingestion / API Intelligence EngineInternet Edge Cloud SimplifiedArchitecture of AI-powered Application Inference result Raw data Back to…
- 21. Ingestion & API Intelligence Engine Internet Edge Cloud Intelligence CurrentTrend PaR of “intelligence” move to the Edge
- 22. But, why AIat the Edge?
- 23. AI at TheEdge Pros: High pemormance: less latency as no data need to go to cloud Works onine: no internet connectivity needed to exchange data with cloud. Beher privacy: data can stays on-device if necessary Eocient power consumption: no need to send large data to cloud Cons: ML model needs to be optimised: optimisation during or post training. Post training optimisation involves model conversion which not always be trivial Challenging deployment & maintenance: new ML model means OTA update is needed for many devices
- 24. So, how canwe achieve AI at the Edge?
- 25. ML-accelerated processor a class ofmicroprocessor designed as hardware acceleration for AI applications, e.g. for neural networks, machine vision and machine learning Example: GPU FPGA ASIC Development Board / Accessories a standalone computing system with ML- accelerated processor (main or co-) as a main or additional development unit Example: Coral DevBoard or USB Accelerator Intel Neural Compute Stick Sipeed Maix Sooware Tools Compilers ML Model Convecer SDK OS Example: TensorFlow & TensorFlow Lite OpenVino nncase SuppoRs Documentation Model Zoo Examples Datasheets Hardware Sooware What do we need?
- 26.
- 28. Object Detection at theEdge What does really happen there?
- 29. But I’m usingA Raspberry Pi (+ Intel NCS 2)
- 30. ML-accelerated processor Optimized deep learning solutionsacross multiple Intel® hardware plarorms Example: CPU (Xeon) FPGA (Arria) ASIC (Movidius) Development Board / Accessories Example: Intel Neural Compute Stick (Movidius) UP Square Board + AI Vision X Dev Kit IEI TANK AIoT Dev Kit Sooware Tools Model Optimizer Inference Engine Model Downloader SuppoRs Documentation Reference Implementation Model Zoo Examples Datasheets Hardware Sooware Intel OpenVino Open Visual Inferencing and Neural Network Optimization hhps://docs.openvinotoolkit.org
- 31. UP Square Board+ AI Vision X Dev Kit IEI TANK AIoT Dev Kit Dev Kit for Edge Intelligence using Intel Hardware
- 32. UP Square Board+ AI Vision X Dev Kit Nvidia Jetson Nano Google Coral Dev Board Raspberry Pi 3 + Intel NCS 2 We can use SBC + NNA
- 33.
- 34. For Low PowerUse Case Use Microcontroller (MCU) Sipeed Maix K210-powered dual core 64-bit RISC-V 400 MHz With/Without WiFi SparkFun Edge 32-bit ARM Cortex-M4F 48 MHz; 1MB Flash; 384KB SRAM BLE 5 Radio SensorTile STM32L476JG 32-bit ARM Cortex- M4 80 MHz; 1MB Flash; 128KB SRAM Accel, Gyro, Compass, Pressure, Microphone sensor BLE 4.2 Radio
- 35. Let me talka bit of Sipeed Maix Disclaimer: I have no asliation with Sipeed (yet)
- 36. Sipeed Maix Go- K210-based AI Board hPps://www.seeedstudio.com/Sipeed-MAix-GO-Suit-for-RISC-V-AI-IoT-p-2874.html
- 37. 0.3-0.5Wahs Better yet… Idle tofull power inference 0.03 waP for sleeping mode
- 38.
- 39. Choosing Network Archfor Image Classiwcation
- 40. KPU DVP K210 Processor Camera Sensor MobileNet Model CPU Classiwcation Result Object MaixPy (MicroPython) But,17MB is still too big! Image Classification using Maix
- 41.
- 42. ML Model Pipelinefor K210 Compress (toco) Optimize (nncase) Train (on your awesome machine or Cloud) .h5 .pb .Ilite .kmodel Copy to MCU via SD Card, or Burn to its Flash Inference (KPU library) Inference Result
- 43. ML Model Pipelinefor AI at The Edge - in General Optimize (compress, remove, replace) Train (on your awesome machine or Cloud) .h5 .pb .cayemodel ONNX Intermediate Representation File Transfer IR gle to Edge device Inference (Edge-optimized Inference library) Inference Result
- 44. Image Classification with yourown dataset What about…
- 45. AIoT-powered Smart TrashManagement Platform We build… by DycodeX
- 46. Sorting Plastic and non-plastictrash using AI-Powered machine vision Compacting Optimizing trash storage Reward Calculate reward point based on trash type Notify & Monitoring Let the waste management official knows upon full capacity or other conditions What it can do
- 47.
- 48. Train Trash ClassificationModel Compress (toco) Optimize (nncase) Train (on your awesome machine or Cloud) .h5 .pb .Ilite .kmodel Copy to MCU via SD Card, or Burn to its Flash Inference (KPU library) Inference Result Trash DataSet (TrashNet) Trash Layers (Based on MobileNet v1)
- 49. KPU DVP K210 Processor Camera Sensor AModel CPU Plastic BoPle Object MaixPy Glass BoPle Other Trash Classiwcations Result Trash Classification using Maix + MaixPy
- 50. AI at theEdge for ESPectro32? Now, what about…
- 51. NB-IoT + GPS/GNSSKit LoRa/LoRaWAN + GNSS Kit Sensors & Actuators Kit (20+ Peripherals) 3G + GNSS + Sensors ESPectro32 Optimized for learning STEM, coding & electronics, also for fast prototyping. An IoT development Board: Low-power, dual-core 240 MHz Microcontroller Built-in WiFi & Bluetooth connectivity 7x7 LED matrix for user intemace Sensors: temperature digital sensor, phototransistor, accelerometer microSD card slot Extensible with Backpacks: NB-IoT/Cellular, LoRa, Sensor Kits, Motor Driver, and more ESPectro32 All products’ electronic board is designed & assembled in-house by 100% Indonesian Backpacks
- 52. Pipeline Train (on yourmachine or Cloud) .nnp wle (Intermediate Representation File) Inference (nnabla C runtime) Inference Result Design (Python, C++) Generate Code (nnabla_cli)
- 53.
- 54. using ESPectro32 andNNABLA MNIST
- 55. GET /index.html POST /upload/wle Inference/predict hand-wriPennumber Display predicted number on LED Matrix Demo Flow
- 56. AI at theEdge for non-image data? Now, what about…
- 57.
- 58. KPU I2C K210 Processor HAR Model (.kmodel) CPU SiPing MaixPy Standing Walking Classiwcations Result ActivityRecognition using Maix + MaixPy IMU Sensor Running Retrain Compress & Optimize .h5 .pb HAR DataSet HAR Layers
- 59. using Keras Train ActivityRecognition Model
- 60.
- 61.
- 63. Intel OpenVino: Docs: hhps://docs.openvinotoolkit.org/ ModelZoo: hhps://github.com/opencv/open_model_zoo Implementations: hhps://soMware.intel.com/en-us/smac-video MicroPython for K210: Sipeed Maix Boards: hhps://www.seeedstudio.com/tag/MAIX.html MaixPy: hhp://github.com/sipeed/MaixPy MicroPython for Kendryte K210: hhps://github.com/loboris/MicroPython_K210_LoBo DycodeX: SMARTernak: hhps://smacernak.com ESPectro32: hhps://shop.makestro.com/product/espectro32-v2/ Other products & solutions: hhps://dycodex.com Azure: Azure Machine Learning: hhps://azure.microsoM.com/en-in/services/machine-learning- service/ Contact me: andri@dycodex.com hhp://github.com/andriyadi hhps://www.slideshare.net/andri_yadi/ Call to Action
- 64. Contact us toget these
- 65.
- 66. makestroid makestroid makestroid makestro.com An Indonesia Platformfor Maker: to “democratize” knowledge, hardware kit, and software to help makers to start making in hardware, to drive into Internet of Things Start Making at Learning Hardware Marketplace Software & Cloud Community RIoT is Makestro’s program for nurturing IoT makers, officially supported by Indonesia’s Ministry of ICT
- 67. Want to put“AI” in “BrAIns”?
- 68. hi@dycodex.com | https://dycodex.com AIoT& Maker Movement enabler Bandung, Indonesia Keep in touch