Downloaded 62 times
Uploaded byNVIDIA Taiwan
Affordable AI Connects To A Better Life
This document discusses making AI more affordable and accessible through techniques like model compression, quantization, and performing inference on edge devices instead of in the cloud. It provides examples of applying these techniques to applications like a Pepper robot performing computer vision tasks and a campus security system using edge devices and Nvidia's Jetson TX1 for real-time intelligent video analysis. The document outlines various approaches to optimize deep learning models for efficient inference on embedded systems with limited memory and computing power in order to bring the benefits of AI to more applications.
More Related Content
Similar to Affordable AI Connects To A Better Life
Affordable AI Connects To A Better Life
- 1. Affordable AI Connects ToA Better Life Bofu Chen, Sep 21, 2016
- 2. Intelligent Gateway Affordable AITechniques Implementation Example: Pepper Robot Example: Campus Security System AGENDA
- 3.
- 4.
- 5.
- 6.
- 7.
- 8. Deep Learning Inference Cat! NoBackpropagation
- 9. Inference Essentials MB Computing TimeMemory Usage Shorten the prediction time is always welcome Device memory is limited, but deep learning model can be huge
- 10. Techniques To MakeAI Affordable
- 11. Inference Researches Weight StorageHardware Usage Reduce weight storage size without sacrificing accuracy Utilize computing components (CPU, GPU, etc.) as many as possible simultaneously Binarized Neural Networks, http://arxiv.org/abs/1602.02830 | XNOR-Net, http://arxiv.org/abs/1603.05279 | DoReFa-Net, https://arxiv.org/abs/1606.06160 | DeepX, http://niclane.org/pubs/deepx_ipsn.pdf
- 12. Approaches Compression Nvidia TensorRT Optimization Throughput Power efficiency Memory usage Keepaccuracy Speed up Low-level speed up
- 13. Nvidia TensorRT Like amodel compiler Production Deep Learning with NVIDIA GPU Inference Engine, https://devblogs.nvidia.com/parallelforall/production-deep-learning-nvidia-gpu-inference-engine/
- 14. Pruning Learning both Weightsand Connections for Efficient Neural Networks, https://arxiv.org/abs/1506.02626
- 15. Quantization How to QuantizeNeural Networks with TensorFlow, https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/how_tos/quantization/index.md DNN is noise tolerable FP16 to INT8 Hardware speedup FPU to ALU
- 16. Inference Without DLFrameworks Likely A compiler intermediate representation for image recognition and heterogeneous computing, http://liblikely.org/
- 17.
- 18. Deep Learning ComputerVision NVIDIA TX1 Pre-trained On Server
- 19. End Devices (Sender) Architecture EndDevices (Receiver) Intelligent Gateway NVIDIA TX1 Ubuntu Tensorflow REST TensorRT gRPC
- 20.
- 21. TensorFlow on TX1 DONE ModelServer Maximize Performance NEXT Inference Optimization on Ubuntu
- 22. Other Attempts Raspberry Pi3 Qualcomm Snapdragon 801 0.9s/img GoogLeNet Real-Time Inception v3
- 23.
- 25. HW Specification 4-core 1.9 GHz 4GB 790 MHz Pepper motherboard specification, http://doc.aldebaran.com/2-4/family/pepper_technical/motherboard_pep.html
- 26. Vision and SpeechLimitations Instead of face identification Keywords instead of NLP FaceRecognition SpeechRecognition
- 27. Cloud Solution Drawbacks CostConnectivityPrivacy Need to ensure bandwidth, stability and latency are good enough Huge amount of image transmission You might want to keep family information locally
- 28.
- 30. Real World GestureRecognition Algorithm
- 44.
- 45.
- 46.
- 47.
- 48.
- 49.
- 55.
- 56. Kinect v2 Update USB Firmware OpenSource Libraries Fix data transmission issue libfreenect2 and pylibfreenect2 make enablement easier MS Kinect v2 on Nvidia Jetson TX1, http://jetsonhacks.com/2016/07/11/ms-kinect-v2-nvidia-jetson-tx1/
- 57.