Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Artificial intelligence on the Edge
Similar to Artificial intelligence on the Edge (20)
More from Usman Qayyum
Recently uploaded
Recently uploaded (20)
Artificial intelligence on the Edge
- 1. 1 AI on the Edge Lifecycle of AI solution to edge computing Usman Qayyum Additional Director (AI) PhD (AI & Robotics) Post-doc (Self-driving Cars) mrusmanqayyum@hotmail.com
- 2. 2 Artificial Intelligence Human Intelligence exhibited by machines
- 4. 4 IOT on Edge-based AI is the most disruptive trend in modern application development
- 6. 6
- 7. 7 Edge-AI is Data-First Application Development
- 11. 11 Selecting AI on Edge
- 12. 12 Edge Computing Applications: Humanity
- 13. 13 Edge Computing Applications: Environment
- 14. 14 What are we going to discuss now ? ● AI algorithms optimize for hardware ○ Quantization ○ Pruning ● Edge Hardware ○ CPU ○ GPU ○ FPGA ○ ASIC ○ Software Toolkit for Edge
- 15. 15
- 16. 16 ● We can use CPU Cache ● No need to read from RAM ● I can run it in a device with smaller memory
- 17. 17 What can I do to optimize? COMPUTE LESS COMPUTE FASTER MOVE LESS DATA
- 18. 18 What can I do to optimize? COMPUTE FASTER ● Increase the speed you compute ○ Increase the clock speed. ● Compute several things at the same time ○ SIMD instructions ○ Multiprocessing
- 19. 19 What can I do to optimize? COMPUTE LESS & MOVE LESS DATA ● Decrease the number of parameters ● Decrease the size of the parameters
- 20. 20 What can I do to optimize? PRUNING Removing branches that are not important for our goal L1 layer regularization Structured pruning by varying regularizer parameters Deep compression
- 21. 21 What can I do to optimize? QUANTIZE Reduce the representation of the data values we use Deep compression Super Nice explanation of 8bit quantization in TF
- 22. 22 LET’S SEE SOME HARDWARE
- 23. 23 CPU to Neuromorphic Computing
- 24. 24 Intel AI on Edge modules
- 25. 25 Qualcomm Snapdragon 845 / Apple A12X
- 26. 26 HiKey 970 ● This is the first HiSilicon SoC with a NPU ● Price 300$
- 27. 27 Sophon Edge Sophon Edge was released in 2018 and it features the SoC Sophon BM1880 which includes a TPU. This SoC can theoretically deliver up to 2TOPS/s Price 129$
- 28. 28 FPGA FPGA is in the middle point between CPU and ASIC The good thing of hardware world: ● Low latency ● Low power consumption The good things from software: ● You can update your hardware ● Cheaper than building in silicon Quite complex to program
- 29. 29 FPGA They are complex to program but... ● Libraries to the rescue! CHaiDNN Library for Xilinx OpenVINO for Intel devices
- 30. 30 Chamaleon96 Cyclone V FPGA ● This board is going to be different compared to the rest because this one has a FPGA instead of a dedicated hardware accelerator ● Price 130$
- 31. 31 FPGA INTEL STRATIX/ARRIA/CYCLONE Series XILINX ZYNQ Series
- 32. 32 Let’s talk about some hardware
- 33. 33 TPU CPU GPU TPU ● It’s an ASIC ● TPUv1 has only 8bit fixed-point support (ONLY USEFUL FOR TRAINING) ● TPUv2 includes 16bit float (NOW WE CAN TRAIN :D )
- 34. 34 Google Coral Dev Board ● A Raspberry Pi look like board but featuring some sort of TPU that allows running NNs in the edge really fast. ● Google Promises real time inference for NNs ● Price $129
- 35. 35
- 36. 36 Mini TPU Google says the next: “Edge TPU enables users to concurrently execute multiple state-of-the-art AI models per frame, on a high-resolution video, at 30 frames per second, in a power-efficient manner.” IT’S in EARLY ACCESS STAGE :( Edge TPU
- 37. 37 Intel Compute Stick 2 USB 3.0 Connection Able to run object detection at ~10fps Compatible with Tensorflow, Caffe2 or ONNX (as long you have the supported layers) V1 ~60€ V2 ~ 100€ but 8x speed up compared to v1.
- 38. 38 1.3 TFLOPS FP32 from the GPU - GPU FP16 is 10.4 TFLOPS. NVIDIA Jetson Xavier GTX1080ti has ~10TFLOPS at FP32
- 39. 39 OpenVino Toolkit by Intel
- 42. 42 AI on Edge (Complete Roadmap)
- 43. 43 Conclusion: Why Edge Computing
- 44. 44 Thanks for your attention. AI on the Edge Usman Qayyum Additional Director (AI) PhD (AI & Robotics) Post-doc (Self-driving Cars) mrusmanqayyum@hotmail.com