Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Embedded Fest 2019. Dov Nimratz. Artificial Intelligence in Small Embedded Systems

48 views

Published on

Majority of IoT solutions use data analysis at the Cloud level, collecting a huge amount of raw data from many thousands of peripherals. What if I told you that you can move from raw data collection to knowledge aggregation by implementing Artificial Intelligence into IoT systems?
During the talk, I will show the benefits of introducing AI at the earliest possible stages, applying the concept of moving from Cloud computing to Fog computing. The basic principle of constructing AIoT systems is the use of the node logic, where a node of the system has to process the provided information in a form of abstract concepts, but not in a form of raw information.
Further, the experience of one device learning and the history of its life cycle can be applied to new models, automatically programming their production cycles for the most efficient use. Actually, IoT solutions should apply AI components at each level of data transfer. Following this approach, the whole system becomes self-optimizing.
Also, during the talk, I will present related case studies and demonstrate a working stand.

Published in: Education
  • Be the first to comment

  • Be the first to like this

Embedded Fest 2019. Dov Nimratz. Artificial Intelligence in Small Embedded Systems

  1. 1. 1 Confidential AIoT from Cloud to Fog Computing Dov Nimratz Senior Solution Architect November 2019
  2. 2. 2 Confidential 2 Agenda 1. About Myself 2. Historical Content 3. Why now 4. IoT and AI synergy 5. AIoT Edge and Cloud Accelerator Parts 6. What to develop
  3. 3. Confidential 3 About Myself Dov Nimartz ● 30+ years in R&D ● 17 years in Israel HighTech ECI, Telrad, RAD, Audiocodes companies ● HW, SW, Mechanical design engineer ● Project & Product Manager ● Business developer for EMEA & CIS ● Solution Architect ● 30+ publications, US patent, 70+ speeches ● Counseling & SW development teaching dov.nimratz@globallogic.com Skype: dovnmr
  4. 4. Confidential 4 Historical Content Analog processors Domain specific models CPU computing GPU computing VPU computing TPU computing CNNNN Optimizations
  5. 5. Confidential 5 Why Now Intel Neural Network Stick • Price $109 • Size 75x27x14 mm • Weight 18 gram • Based on Myriad X VPU • 16 video CPU • MobileNet v2 20 fps • TensorFlow, Cafee • Current up to 40 mA NVIDIA Jetson Nano • Price $100 • Size 95x76x30mm • Weight 136 gram • 64bit quad-core A57 • 128-core Nvidia GPU • MobileNet v2 12 fps • Current up to 1220 mA Google TPU Accelerator • Price $75 • Size 65x30x8 mm • Weight 12 gram • Google Edge TPU • MobileNet v2 40+ fps • TensorFlow Lite • Current up to 20 mA Google Coral Dev Board • Price $150 • Size 85x56x20mm • Weight 78 gram • NXP i.MX 8M SOC • Google Edge TPU • MobileNet v2 100+ fps • TensorFlow Lite • Current up to 960 mA
  6. 6. Confidential 6 Platforms Benchmark Resume: • Absolute winner is Coral Dev Board. • Intel NN Stick is effective at face recognition related projects. • Coral USB Accelerator is the cheapest solution with a good performance. • NVIDIA Jetson Nano is not recommended for use.
  7. 7. Confidential 7 IoT and AI Synergy Collect StoreExecute Analyse Process IoT Collect StoreExecute Analyse Process AI
  8. 8. 8 Confidential 8 Edge IoT System Concept Cloud part consists of: ● different but similar components ● data ETL core ● different presentation layers. But, we may have similar sensors or Edge Layer: AI Embedded in the Edge Physical Edge Local Network Gateway Wide Network Security Middleware ETL Presentation Notification Configuration BackEndEdge Ad Hoc / Mesh M2MM2P Big Data, Analytic
  9. 9. 9 Confidential 9 Edge Layer Watchdog State Mashine Main Logic Sensor Configuration Collected data Local Network Layer Battery Edge Layer Edge Part First business layer responsible for: ● Gather information ● Abstraction generation ● Neighbors redundancy communication
  10. 10. 10 Confidential 10 Middleware Layer Load Balancers Queues Subscribers Middleware Part ● Spread information by load ● Provide asynchronous call computation
  11. 11. 11 Confidential 11 ETL Layer Extract Transform ArchiveBucket OLAP OLTP SQL Notifications If the data does not have a strict scheme (table columns), then it is stored in NoSQL databases. SQL databases have two types: 1. OLTP (Online Transactional Processing) 2. OLAP (Online Analytical Processing) NoSQL
  12. 12. 12 Confidential 12 Edge IoT System Concept Cloud part consists of ● different but similar components ● data ETL core ● different presentation layers. But, we may have similar sensors or Edge Layer: AI Embedded in the Edge Physical Edge Local Network Gateway Wide Network Security Middleware ETL Presentation Notification Configuration BackEndEdge Ad Hoc / Mesh M2MM2P Big Data, Analytic
  13. 13. 13 Confidential 13 Edge AIoT System Concept Changes from IoT model ● Universal AI Edge sensor ● ML models DB ● IFTTT State machine (Configurable) ● Additional parameters in LvM2M modes AI Edge Local Network Gateway Wide Network Security Middleware ETL Presentation Notification Configuration BackEndEdge Ad Hoc / Mesh M2MM2P FSM AI Edge ML Models IFTTT
  14. 14. 14 Confidential 14 Fog Computing AI-Based Architecture Flow ... ... ... Abstract data New ML model Raw data request and send Model tuning ● Exchanging knowledge in place of data ● Uploading abstract ● Downloading knowledge ● Self improvement ● Raw data upload by a request from the upper layer
  15. 15. 15 Confidential 15 ML Models Optimization for Edge AI Frameworks: ● TensorFlow ● TensorFlow Lite ● Caffe Optimizations: ● Quantization ● Tensor Fusion MobileNet SSD SqueezeNet Inception Models: ● MobileNet SSD v1 & v2 ● SqueezeNet 1.0, SqueezeNet 1.1 ● Inception v1 & v2 & v3 3 & v4 Libraries: ● OpenCV ● OpenVINO QuantizationTensorRT Optimisation next input 3x3 CBR 3x3 CBR 1x1 CBR 1x1 CBR max pool next input input tensor Expansion layer Depthwise layer filter the data Projection layer uncompress the data compress the data outpt tensor Min Input (float) Min Max Quantize QuantizedRelu Dequantize Output (float) Ma x Eight Bit Min Ma x Eight Bit maxpool/2 conv1 fire2 fire3 fire4 fire5 fire6 fire7 fire8 fire9 fire10 softmax maxpool/2 globalavgpool maxpool/2 96 128 128 256 256 384 384 512 512 1000
  16. 16. 16 Confidential 16 IFTTT Final State Machine ● Configurable FSM ● Generic data concept ● Visual simulation ● Visual monitoring States list Objects list Events list Action list
  17. 17. 17 Confidential AIoT Edge and Cloud Accelerator Parts
  18. 18. 18 Confidential 18 Edge AIoT Accelerator Module Battery AI Device Display Camera ● A fully standalone video processing device ● Does not require cloud or mobile device connection for operation presenting ● Contains replaceable parts ● Single hardware for any IoT solution demonstration ● Contains port for extended sensors and Ethernet with PoE ● ML models can be downloaded or updated from the cloud to support a specific IoT solution demonstration
  19. 19. 19 Confidential 19 AIoT Accelerator Cloud Part Reaction Objects AI Camera Vision AB SM AI camera classifies objects ● Object attributes: ○ ID, probability, location, and state ● AB SM events: ○ Object ID change ○ Time of the day ○ Calendar ○ Public events ● AB SM reactions: ○ Change of state ○ Giving out notifications ○ Control of accelerators ○ Change Object attributes
  20. 20. 20 Confidential 20 AIoT Accelerator Block Diagram Object Action Location Detection Event generation FSM updateActions Actions Internal ● Update Edge ML nodel ● Start record stream ● Take an image shoot ● Assign ID to object/location ● Change object attribute External ● Make a call ● Control switch/attenuator ● Send a sound message ● ETL action
  21. 21. 21 Confidential What to Develop
  22. 22. 22 ConfidentialConfidential ML Models
  23. 23. Confidential 23 LvM2M Data Model Upgrade ● Objects array ● Probability information ● Correlation data ● Neighbor sensor data
  24. 24. Confidential 24 IFTTT STM Design ● Templates from different solution ● Custom Customization ● Visual simulation ● Visual history and action logic ● Ability for a third party device integration ● API for the external applications
  25. 25. 25 Confidential Questions? Dov Nimratz Senior Solution Architect dov.nimratz@globallogic.com +380-91-333-3300
  26. 26. 26 Confidential Server Client Demonstration
  27. 27. 27 Confidential Thank you Dov Nimratz Senior Solution Architect dov.nimratz@globallogic.com +380-91-333-3300

×