Downloaded 23 times
More Related Content
Similar to Artificial intelligence on the Edge
Artificial intelligence on the Edge
- 1. 1 AI on theEdge Lifecycle of AI solution to edge computing Usman Qayyum Additional Director (AI) PhD (AI & Robotics) Post-doc (Self-driving Cars) mrusmanqayyum@hotmail.com
- 2. 2 Artificial Intelligence Human Intelligenceexhibited by machines
- 3.
- 4. 4 IOT on Edge-basedAI is the most disruptive trend in modern application development
- 5.
- 6.
- 7. 7 Edge-AI is Data-FirstApplication Development
- 8.
- 9.
- 10.
- 11.
- 12. 12 Edge Computing Applications:Humanity
- 13. 13 Edge Computing Applications:Environment
- 14. 14 What are wegoing to discuss now ? ● AI algorithms optimize for hardware ○ Quantization ○ Pruning ● Edge Hardware ○ CPU ○ GPU ○ FPGA ○ ASIC ○ Software Toolkit for Edge
- 15.
- 16. 16 ● We canuse CPU Cache ● No need to read from RAM ● I can run it in a device with smaller memory
- 17. 17 What can Ido to optimize? COMPUTE LESS COMPUTE FASTER MOVE LESS DATA
- 18. 18 What can Ido to optimize? COMPUTE FASTER ● Increase the speed you compute ○ Increase the clock speed. ● Compute several things at the same time ○ SIMD instructions ○ Multiprocessing
- 19. 19 What can Ido to optimize? COMPUTE LESS & MOVE LESS DATA ● Decrease the number of parameters ● Decrease the size of the parameters
- 20. 20 What can Ido to optimize? PRUNING Removing branches that are not important for our goal L1 layer regularization Structured pruning by varying regularizer parameters Deep compression
- 21. 21 What can Ido to optimize? QUANTIZE Reduce the representation of the data values we use Deep compression Super Nice explanation of 8bit quantization in TF
- 22. 22 LET’S SEE SOMEHARDWARE
- 23. 23 CPU to NeuromorphicComputing
- 24. 24 Intel AI onEdge modules
- 25. 25 Qualcomm Snapdragon 845/ Apple A12X
- 26. 26 HiKey 970 ● Thisis the first HiSilicon SoC with a NPU ● Price 300$
- 27. 27 Sophon Edge Sophon Edgewas released in 2018 and it features the SoC Sophon BM1880 which includes a TPU. This SoC can theoretically deliver up to 2TOPS/s Price 129$
- 28. 28 FPGA FPGA is inthe middle point between CPU and ASIC The good thing of hardware world: ● Low latency ● Low power consumption The good things from software: ● You can update your hardware ● Cheaper than building in silicon Quite complex to program
- 29. 29 FPGA They are complexto program but... ● Libraries to the rescue! CHaiDNN Library for Xilinx OpenVINO for Intel devices
- 30. 30 Chamaleon96 Cyclone VFPGA ● This board is going to be different compared to the rest because this one has a FPGA instead of a dedicated hardware accelerator ● Price 130$
- 31.
- 32. 32 Let’s talk aboutsome hardware
- 33. 33 TPU CPU GPU TPU ● It’s an ASIC ● TPUv1 has only 8bit fixed-point support (ONLY USEFUL FOR TRAINING) ● TPUv2 includes 16bit float (NOW WE CAN TRAIN :D )
- 34. 34 Google Coral DevBoard ● A Raspberry Pi look like board but featuring some sort of TPU that allows running NNs in the edge really fast. ● Google Promises real time inference for NNs ● Price $129
- 35.
- 36. 36 Mini TPU Google saysthe next: “Edge TPU enables users to concurrently execute multiple state-of-the-art AI models per frame, on a high-resolution video, at 30 frames per second, in a power-efficient manner.” IT’S in EARLY ACCESS STAGE :( Edge TPU
- 37. 37 Intel Compute Stick2 USB 3.0 Connection Able to run object detection at ~10fps Compatible with Tensorflow, Caffe2 or ONNX (as long you have the supported layers) V1 ~60€ V2 ~ 100€ but 8x speed up compared to v1.
- 38. 38 1.3 TFLOPS FP32from the GPU - GPU FP16 is 10.4 TFLOPS. NVIDIA Jetson Xavier GTX1080ti has ~10TFLOPS at FP32
- 39.
- 40.
- 41.
- 42. 42 AI on Edge(Complete Roadmap)
- 43. 43 Conclusion: Why EdgeComputing
- 44. 44 Thanks for yourattention. AI on the Edge Usman Qayyum Additional Director (AI) PhD (AI & Robotics) Post-doc (Self-driving Cars) mrusmanqayyum@hotmail.com