"Achieving High-Performance Vision Processing for Embedded Applications with Qualcomm SoC Platforms," a Presentation from Qualcomm
•
1 like•201 views
For the full video of this presentation, please visit: https://www.embedded-vision.com/platinum-members/qualcomm/embedded-vision-training/videos/pages/may-2018-embedded-vision-summit For more information about embedded vision, please visit: http://www.embedded-vision.com Sahil Bansal, Senior Director of Product Management at Qualcomm, presents the "Achieving High-Performance Vision Processing for Embedded Applications with Qualcomm SoC Platforms" tutorial at the May 2018 Embedded Vision Summit. Advancements in machine learning are making it possible to equip devices such as connected cameras, robots, drones and smart home solutions with improved on-device vision processing and analytics capabilities. This talk addresses how a hybrid approach, combining deep learning with traditional computer vision and utilizing Qualcomm Technologies System-on-Chips (SoCs) and solutions can deliver significant performance and power-efficiency improvements for embedded applications requiring vision processing. Bansal presents the benefits and trade-offs, use cases and results from concrete implementations using this hybrid approach.
Recommended
More Related Content
Similar to "Achieving High-Performance Vision Processing for Embedded Applications with Qualcomm SoC Platforms," a Presentation from Qualcomm
Similar to "Achieving High-Performance Vision Processing for Embedded Applications with Qualcomm SoC Platforms," a Presentation from Qualcomm (20)
More from Edge AI and Vision Alliance
More from Edge AI and Vision Alliance (20)
Recently uploaded
Recently uploaded (20)
"Achieving High-Performance Vision Processing for Embedded Applications with Qualcomm SoC Platforms," a Presentation from Qualcomm
- 1. Sahil Bansal, Senior Director of Product Management Qualcomm Technologies, Inc.
- 2. Computer vision is empowering a broad set of applications Mobile imaging VR/AR Drones Robotics IP Camera Automotive
- 3. Driving advances in computer vision Simultaneous localization and mapping Touch-to-track Face detection and recognition Object detection and classification 6-DOF positional tracking
- 4. Versatility across use cases Better utilization of big data Greater accuracy Ease of development
- 5. Process data closest to the source, complement the cloud On-device intelligence is paramount Privacy Reliability Low latency Efficient use of network bandwidth
- 6. Higher accuracy requires greater processing capabilities 50 55 60 65 70 75 80 0 5 10 15 20 4025 30 35 155M125M95M65M35M 5MBN-AlexNet AlexNet BN-NIN ENet GoogLeNet ResNet-18 ResNet-34 ResNet-50 Inception-v3 Inception-v4 ResNet-101 ResNet-152 Top-1accuracy[%] Operations [G-Ops] Size of circles proportional to number of network parameters.
- 7. Two Sizes of Buffer • Smaller buffer for DL wrt object detection/Classification use-case • Larger buffer for CV wrt tracking use-case Greater control detection and tracking in distance Lite object tracker (LOT) On-going tracking task? Tracking reliability evaluation (TRE) Add new objects or remove obsoletes DL based object detector (OD) Time for detection? Strong object tracker (SOT) Visualize result Yes No Yes No Associate objects between outputs from OD/SOT and LOT
- 8. ROI determination Two Sizes of Buffer • Scaling factor and location of the smaller working buffer of DL for object detection can be based on feedback from the tracker or other clues • Larger buffer for CV wrt tracking use-case Greater control detection and tracking in distance Lite object tracker (LOT) On-going tracking task? Tracking reliability evaluation (TRE) Add new objects or remove obsoletes DL based object detector (OD) Time for detection? Strong object tracker (SOT) Visualize result Yes No Yes No Associate objects between outputs from OD/SOT and LOT
- 9. Downscale Partition Face Merge Person Ball Person Person Person Bal l Phone Person Person Person Person Ball Face Phone
- 10. Tradeoff between Speed and Detectable Distance Speed: 1 X Detectable Distance: 1 X Speed: 1X (per 5 frames) Detectable Distance: 2 X Person Person Person Person Face Phone Person Ball Ball
- 11. • •
- 12. Heterogeneous computing key for on-device intelligence Designed to deliver performance and efficiency improvements Broad portfolio of SoCs addresses different levels of performance and price points Memory Qualcomm Spectra ISP Qualcomm Adreno GPU CPU Qualcomm Snapdragon LTE Modem Display Processor (DPU) Qualcomm Hexagon DSP Video Processor (VPU)
- 13. Choose the core to match the user experience 8x Performance* Qualcomm® Kryo CPU Adreno GPU Hexagon DSP Energy Efficiency* 25x 4x 8x
- 15. Purpose-built 10nm SoCs for next-gen robots, smart cameras and smart home Qualcomm AI Engine • Hardware + Software to accelerate on-device AI • Up to 2.1 TOPS for DNN inferences Superb image with our most powerful camera processor ever • Dual 14-bit Spectra™ ISP supporting dual 16 Mpix sensors • Up to 4K video @ 60 FPS, or 5.7K @ 30 FPS • Staggered HDR, electronic image stabilization, de-warp, de-noise and more Heterogeneous compute and key features • Up to 8x Kryo™ 360 CPU, Adreno™ 615 GPU, Hexagon 685 Vector Processor • Up to WQHD touch display, 2x2 11ac Wi-Fi®, Bluetooth® 5.1, advanced audio • Hardware-based security
- 16. Ecosystem • OEMs/ODMs • ISVs • Dev Tools • Snapdragon Neural Processing Engine • Android NN • Hexagon NN Hardware Frameworks • Caffe/Caffe2 • TensorFlow/TensorFlow Lite • ONNX Applications & FeaturesINT8 precision networks FP32 and FP16 precision networks FP32 and INT8 8bit precision networks with Qualcomm Artificial Intelligence Engine
- 17. • Sampling now • VR 360 camera reference design available today • Industrial security camera reference design expected 2H’18 • Ecosystem of technology providers www.qualcomm.com/IoT Vision Intelligence Platform
- 18. Advanced camera processing, powerful machine learning, and computer vision at the edge will allow new applications that push the boundaries of our connected world. AI in here. And everywhere. Intelligent cameras monitor and track what’s inside. Neural networks watch and learn your preferences. Ubiquitous connectivity helps you avoid running out of critical food supplies Refrigerator Inventory Chicken Bell Peppers Soy Sauce Purple Basil Raspberries Milk (level is low) Local at the edge processing means low latency and max efficiency.
- 19. Instant inventory with live images from inside your refrigerator combined with clever neural networks tuned to keep essential groceries in your home and meals on the table. Easy. Smarter appliances deliver cooked to perfection every time. It learns how you like your food prepared. Integrated machine learning will take the guesswork out of cooking. With smart assistants to keep things coordinated, get ready to enjoy perfect culinary bliss. Dinner made simple.
- 20. In Your Living Room • Voice/Face Recognition knows between you and other members of the family • Home Control and Automation for intelligent personalized lighting and climate control • Home Security video surveillance and intelligence to better protect your home
- 21. Sally Barnes Status: Active Unknown Status: Sign in ? Allie Oope Status: Active Fred Stone Status: Active Amanda Hogen Status: Active Enterprise Security • Improve security with automated AI screening using people detection, facial recognition and facial detection • Unknown faces are automatically flagged for security assessment Unknown Status: Sign in ?
- 22. Intelligent Smart Cities • Scene Classification • License Plate Recognition • Target Sound Detection
- 23. Thank You