“Designing the Next Ultra-Low-Power Always-On Solution,” a Presentation from Cadence
•
0 likes•5 views
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2022/06/designing-the-next-ultra-low-power-always-on-solution-a-presentation-from-cadence/ Amol Borkar, Director of Product Management and Marketing for Tensilica Vision and AI DSPs at Cadence, presents the “Designing the Next Ultra-Low-Power Always-On Solution” tutorial at the May 2022 Embedded Vision Summit. Increasingly, users expect their systems to be ready to respond at any time—for example, using a voice command to launch a music playlist. System designers have traditionally relied on classical signal processing techniques on simple microcontrollers to implement such features. Today, as more and more devices incorporate cameras and other types of sensors, there’s a growing desire to enable “always-on” functionality using more than just wake words—for example, so that a doorbell camera can wake up when a person approaches. These enhancements require clever AI algorithms that can reliably detect events of interest. And, to enable “always-on” capability, these sophisticated algorithms often must be implemented with ultra-low power consumption, especially for battery-powered devices. In this presentation, Borkar shares the trends his company has observed in always-on features and applications and highlights the latest additions to the Cadence Tensilica processor portfolio that address the needs of ultra-low-power, always-on applications.
Recommended
Recommended
More Related Content
Similar to “Designing the Next Ultra-Low-Power Always-On Solution,” a Presentation from Cadence
Similar to “Designing the Next Ultra-Low-Power Always-On Solution,” a Presentation from Cadence (20)
More from Edge AI and Vision Alliance
More from Edge AI and Vision Alliance (20)
Recently uploaded
Recently uploaded (20)
“Designing the Next Ultra-Low-Power Always-On Solution,” a Presentation from Cadence
- 1. Designing the Next Ultra- Low-Power Always-On Solution Amol Borkar Product Marketing Director, IPG TIP Cadence Design Systems, Inc.
- 2. Cadence Tensilica IP: Business Overview and Recent Success TENSILICA® CUSTOMERS >40B Processors SHIPPED SEMICONDUCTORS 19 19 of the Top 20 SEMICONDUCTOR VENDORS USE TENSILICA GLOBAL ECOSYSTEM 200+ ECOSYSTEM PARTNERS DSP LICENSING REVENUE #1 DSP IP LICENSING REVENUE 2 © 2022 Cadence Design Systems, Inc. All rights reserved.
- 3. Processing and Sensor Trends Automotive, robotics, drones, mobile, and AR/VR Computational Imaging Algorithms 3D Point Cloud/Capture Large Number of Image Sensors/High FPS/High Resolution AR/VR Drone Automotive Mobile Robotics Different Types of Sensors: Image Sensors/Lidar/Radar 3D Sensors (TOF), Image Sensors Sensor Fusion Ultra-Low-Power Computer Vision/AI and Always-On Island NEW 3 © 2022 Cadence Design Systems, Inc. All rights reserved.
- 4. Ultrasonic sensors • Protocols or methodologies to “wake up” the device has been around for a long time and is being used in many industries • Various options for input sources: HDMI, GPIO, Gyroscope… • Using microphone and ultrasonic sensors has been prominent in the past few years • Proliferated through AI for such usages with TFLm and TinyML • Interest in using cameras for “visual wake words” • People detection, face detection, gestures, etc. • Emphasis on vision and speech-based processing for embedded/IoT devices (some ultrasonic) Always-On / Wake-Up Processing HDMI Ethernet GPIO Microphone Camera / Image sensors Gyroscope 4 © 2022 Cadence Design Systems, Inc. All rights reserved.
- 5. • Lightweight algorithms that constantly run or execute on device, typically as a precursor to more accurate (more compute-intensive) algorithms • Also known as low power or standby mode or AON • Applications • Ways to Accomplish This Always-On Primer 5 © 2022 Cadence Design Systems, Inc. All rights reserved. IoT/Battery Powered Devices Face Detection Device Unlock / Authenticate Gesture Control Under Disp. Fingerprint Voice Wake Words AI Visual Wake Word person Video Doorbell Gesture Recognition
- 6. • Basic functionality: • Continuously running or actively monitoring on the embedded device • Process inputs from cameras, microphones or other sensors in real-time (e.g., 5 – 20 fps) • Capable of handling traditional algorithms and lightweight neural networks • Ultra-low-power / energy consumption (e.g., <1 mW) • Introduction of “lite” networks for always-on usages: • Designed to run on microcontroller class of devices (e.g., TensorFlow Lite for Micro) • Common problems in always-on domain • Limited availability on public domain (not present in most common model zoos) • Expectation that model zoo networks have low energy consumption on always-on processors Always-On Requirements Typical Networks Total MAC count (MAC/Inf) Model Zoo Network AON Candidate Network Ratio Face / Person / Object Detection 30 G 7 M 430x Speech / Voice 200 M 20 K 10000x Gesture 10 G 4 M 2500x Networks for AON processors are significantly smaller than typical model zoo networks *Approx values, does not depict any specific network(s) *Resnet50 vs TinyML Image Classification Networks 100 Layers 12 Layers 6 © 2022 Cadence Design Systems, Inc. All rights reserved.
- 7. 3 Stages of Device Wake / Unlock Process / Unlock / Wake Is there something? What is it? Where is it? Process some more Localize and Authenticate Detect 7 © 2022 Cadence Design Systems, Inc. All rights reserved.
- 8. Process / Unlock / Wake Localize and Authenticate Detect 3 Stages of Device Wake / Unlock Sensor Ultrasonic, Microphone, Camera, Fingerprint scanner Microphone, Camera Ultrasonic, Microphone, Camera, Touchscreen and more AI Compute Requirement s < 250GOPs 250 GOPs – 2 TOPs > 2 TOPs Processor mCU or LP DSP DSP, AI Engine, HWA CPU, DSP, GPU, AI Engine, HWA Key Features Ultra low power / energy Area Low power / energy Area Typical Applications Keyword Spotting, Person/Face Detection, Visual Wake Words, Gestures Automatic Speech Recognition, Person / Face recognition and identification, refined gesture processing Pose / Gait recognition, 3D Face, Speech recognition, Gaming SW Tools/Fwk Bare-metal, TFLm TensorFlow, TFLite, Pytorch, ONNX, Imaging, CV etc. Too many to list Ultrasonic, Microphone, Camera, Fingerprint scanner < 250 GOPs mCU or LP DSP Ultra low power / energy Area Keyword Spotting, Person/Face Detection, Visual Wake Words, Gestures Bare-metal, TFLm Detect Target segment for always-on 8 © 2022 Cadence Design Systems, Inc. All rights reserved.
- 9. Focusing on Detection Stage Market Classification Entry-Level / Base Mid-Range Premium Sensor(s) Ultrasonic, Microphone, Low res. camera Microphone or Camera or others Microphone, Camera and others Typical Applications Proximity detection Keyword Spotting, Person / Face detection … Keyword Spotting, Person / Face Detection … Benefits Best Area, Power, Energy Simplified algorithms Leverage cutting edge AI algorithms Higher resolution for identification Area saving with IP repurpose Scalable and future-ready Limitations X Not enough res for user identify X Limited AI capabilities X Able to run only one modality (Keyword Spotting, or Face Detection…) X Increased power consumption Lots of ongoing efforts Emerging markets Proximity detection Keyword Spotting, Person / Face detection … Keyword Spotting, Person / Face Detection … IP RE-PURPOSE 9 © 2022 Cadence Design Systems, Inc. All rights reserved.
- 10. Architectural Considerations: Discrete Data Paths High-Performance Vision DSP / AI Engine High-Performance Audio DSP Vision AON Processor Audio AON Processor Detect Localize and Authenticate Process / Unlock / Wake AON Subsystem Discrete IP to handle vision and audio AON workloads 10 © 2022 Cadence Design Systems, Inc. All rights reserved.
- 11. Architectural Considerations: Combined Data Paths AON Processor High-Performance Vision DSP / AI Engine High-Performance Audio DSP AON Subsystem • One IP handles both vision and audio AON workloads • Repurpose existing IP Detect Localize and Authenticate Process / Unlock / Wake 11 © 2022 Cadence Design Systems, Inc. All rights reserved.
- 12. • There is a need to do always-on processing (AON) • Area-efficient processor with ultra-low power consumption • Operate with extremely small memory footprint i.e., no DDR • Trending towards discrete or single IP to do multi-modal processing • Processing inputs from camera, microphone, ultra-sonic sensor, etc. • Capable of computer vision, AI, speech, sound, FFTs, etc. • Fortunately, Tensilica® has an answer … What Have We Learned So Far? (AKA Requirements) 12 © 2022 Cadence Design Systems, Inc. All rights reserved.
- 13. Selection-Based on Dominant Processing Tensilica® Vision DSP Applications Dominant Processing Processor Choice Sensors 13 © 2022 Cadence Design Systems, Inc. All rights reserved. Tensilica® HiFi DSP Image / Camera Dominant Processing Audio / Speech Dominant Processing
- 14. Leading Solution for Low-Energy Always-On Scalable, Configurable, and Extensible Architecture 128-bit SIMD Up to 128 MAC performance Supports TIE for custom instructions Tensilica® Vision P1 DSP Best-In-Class Low-Energy DSP for Vision and AI DSP Efficient Design 1/3 area and power of Vision P6 DSP Enables Compelling Use Cases Always-on processing Face detection Smart image sensor Foveated rendering Proximity detection Keyword spotting Scalability Based on very successful Vision DSP architecture N-way programming model Fast TTM Leverages existing Vision and AI libraries, tools, frameworks, SDKs for fast development 14 © 2022 Cadence Design Systems, Inc. All rights reserved.
- 15. 15 © 2022 Cadence Design Systems, Inc. All rights reserved. Always-On Face Detection (1) CPU or App Processor Good (CPU) CPU continuously performs face detection and recognition algorithms CPU recognizes face and unlocks phone Example: Phone with CPU only X CV+NN algorithms on CPU are not very efficient Detect, Recognize, Authenticate
- 16. Better (CPU + Vision DSP) Vision DSP continuously performs face detection and recognition algorithms Vision DSP recognizes face and wakes up host CPU to unlock phone Example: Phone with CPU + Vision DSP X DSP is performing face detection when no face is present Always-On Face Detection (2) Tensilica® Vision DSP or NPU CPU or App Processor Detect, Recognize, Authenticate 16 © 2022 Cadence Design Systems, Inc. All rights reserved.
- 17. Best (CPU + Vision DSP + Vision P1) Vision P1 DSP continuously performs simplified face detection algorithms Wakes up Vision DSP to do accurate face detection and recognition Vision DSP recognizes face and wakes up host CPU to unlock phone Best energy efficiency: Vision DSP and CPU are only woken up as needed Always-On Face Detection (3) Tensilica® Vision P1 DSP Tensilica Vision DSP or NPU CPU or App Processor There is a face • Low Resolution • ISP Bypass • Bayer / Mono Detect, Recognize, Authenticate 17 © 2022 Cadence Design Systems, Inc. All rights reserved.
- 18. Always-On Face Detection with Secure Mode Tensilica® Vision P1 DSP Tensilica Vision DSP or NPU CPU or App Processor There is a face Detect, Recognize, Authenticate Secure Mode Boundary Secure Boot Camera is memory mapped to secure zone (TEE) Secure Boot 18 © 2022 Cadence Design Systems, Inc. All rights reserved.
- 19. TinyML v0.5 Benchmark Comparison Inf / Sec Visual Wake Word Closest Comp Vision P1 Inf / Sec Anomaly Detection Closest Comp Vision P1 Inf / Sec Keyword Spotting Closest Comp Vision P1 Inf / Sec Image Classification Closest Comp Vision P1 7X 4X 3X 5X Energy (uJ) Visual Wake Word Closest Comp Vision P1 Energy (uJ) Anomaly Detection Closest Comp Vision P1 Energy (uJ) Keyword Spotting Closest Comp Vision P1 Energy (uJ) Image Classification Closest Comp Vision P1 300X 125X 300X 60X Tensilica® Vision P1 DSP processes vision and audio workloads for always-on Energy efficient and customizable to run at different frequencies <1 mW always-on solution Source: https://mlcommons.org/en/inference-tiny-05/ RTL simulated on P&R netlist 19 © 2022 Cadence Design Systems, Inc. All rights reserved.
- 20. Other Vision Capabilities 20 © 2022 Cadence Design Systems, Inc. All rights reserved. Smart Image Sensor + Normal Image (Pre/Post ISP) Smart Sensor Processing Cat Metadata Region of Interest Crop Bounding Box Segmentation Gaze direction used for foveated rendering Tensilica Vision P1 DSP Headset with pupil camera Cameras capture pupil images Detects pupil center and gaze direction Foveated Rendering Tensilica® Vision P1 DSP
- 21. One Last Thing… 2022 Embedded Vision Summit • Tensilica® Vision P1 DSP Always-On FPGA Demo • Visual Wake Word + Keyword Spotting Demo on same IP • Other Vision and AI Demos Come visit Cadence booth located at 317 Sources: MLCommon Tiny Networks Performance https://mlcommons.org/en/inference-tiny-05/ Always-On Market Trend https://www.grandviewresearch.com/industry- analysis/light-sensor-market 21 © 2022 Cadence Design Systems, Inc. All rights reserved.
- 22. © 2022 Cadence Design Systems, Inc. All rights reserved worldwide. Cadence, the Cadence logo, and the other Cadence marks found at https://www.cadence.com/go/trademarks are trademarks or registered trademarks of Cadence Design Systems, Inc. Accellera and SystemC are trademarks of Accellera Systems Initiative Inc. All Arm products are registered trademarks or trademarks of Arm Limited (or its subsidiaries) in the US and/or elsewhere. All MIPI specifications are registered trademarks or service marks owned by MIPI Alliance. All PCI-SIG specifications are registered trademarks or trademarks of PCI-SIG. All other trademarks are the property of their respective owners.