Optimize multimedia with Qualcomm Hexagon SDK

Qualcomm® Hexagon™ SDK
Optimize Your Multimedia Solutions
Jack May, Senior Staff Engineer
Qualcomm Technologies, Inc.
Qualcomm Hexagon is a product of Qualcomm Technologies, Inc.

4
Designed to easily utilize the capabilities of the Hexagon DSP
Uses the ability to dynamically load and execute code on the Hexagon DSP
Helps reduce development effort and promotes a rapid development cycle
Applications include
− General compute
− Audio
− Voice
− Imaging
− Computer vision
Overview
What is the Hexagon SDK?

5
Provide customization of the Hexagon DSP without rebuilding the static image
− Remote procedure call mechanism between the HLOS and the Hexagon DSP
− Dynamically loadable objects loaded at runtime.
− Dynamic module interfaces to the Audio/Voice/.. Frameworks
Provide multiple debugging options
− USB based debugger
− Diagnostic logging
Provide unit and system level test frameworks and capabilities
Provide both in-simulation and on-target environments
Goals

6
Windows installer
Installs the SDK components (docs, examples, tools, …)
Results in a complete, self-contained development environment
Minimizes environment changes that may impact other tools or installations
Multiple SDK versions may be installed side by side
Downloads 3rd party dependencies
No global PATH or environment variable modifications
Dependencies are self-contained and local to the SDK
− Except
− Hexagon Tools
− Python
Linux support on the roadmap
Installation

8
Fast Remote Procedure Calls (FastRPC)
Dynamic loading
Remote debugger
Eclipse plugin
Hardware development platforms
Optimized Hexagon libraries
Real-time Operation System (RTOS)
Comprehensive tool chain
Diagnostic logging
Notable features
What is in the Hexagon SDK?

9
Remote procedure calls between the applications processor and the Hexagon DSP
Fast
− Synchronous
− Zero copy
− Cache coherent
Functions called on the Hexagon DSP may reside in dynamically loaded code
Maintains cache coherency for input and output buffers
Calling HLOS threads handled by corresponding threads on the Hexagon DSP
Threads are automatically cleaned up when HLOS process exits
Notable features: FastRPC

10
DSPApplications processor
Notable features: FastRPC
In-process
Transport
Application calculator library
calculator_skel
calculator_sum()
#include "calculator.h"
int calculator_sum(
const int* vec,
int vecLen,
int64* res)
{
int ii = 0;
*res = 0;
for(ii = 0; ii < vecLen; ++ii) {
*res = *res + vec[ii];
}
return 0;
}
calculator_sum()
calculator_stub
calculator_sum()
Sample code subject to SDK license agreement. Available at developer.qualcomm.com

11
Notable features: FastRPC: Performance
Size of buffer 8074 in (us) 8074 out (us)
0 62
32 kB 80 90
64 kB 84 94
128 kB 85 106
1 MB 177 211
4 MB 261 212
8 MB 377 303
16 MB 612 538
− Measured with Application and DSP processor clocks at max
− 8074 DragonBoard™ from Intrinsyc
− No change in data and DSP returns immediately
− These are minimum times, and does not include RAM to cache sync times (as if prefetched)
Source: Qualcomm Technologies, Inc. internal testing

12
Allows for the addition of code/data into the Hexagon DSP at runtime
Faster development cycles, only have to build your shared object
Code/data is built into shared object files (.so)
− Executable and Linkable Format (ELF)
− Stored on the HLOS’s file system and read by the Hexagon DSP when loaded
Functions and variables become accessible to the static DSP image once the shared
object is loaded
Native support for shared objects in the audio and voice frameworks
Notable features: Dynamic Loading

13
lib.so
Fxn()
Notable features: Dynamic Loading
HLOS File System
lib.so
Fxn()
aDSP Static Image
h = dlopen(“lib.so”);
pFxn = dlsym(h, “Fxn”);
pFxn();
dlclose(h);
lib.so
Fxn()

14
Easier to create, build, run, test, and debug Hexagon modules
Project templates
Unit and integration testing
Simulator and on-target debugging
Syntax aware and highlighting for Hexagon assembly, C/C++, and IDL
Custom views
− Connected devices
− Profiler
− Resource analyzer
− Audio module database
− Logging
Notable features: Eclipse IDE plugin

15
USB based software, no hardware JTAG required
Industry standard debug tools (GDB with LLDB on roadmap)
Integrated with the Hexagon SDK Eclipse plugin
Single or all thread stop debugging
Simulator and on-target debugging
Notable features: Remote debugger

16
Quick development cycle from picking up the Hexagon SDK to running your code on-target
Affordable hardware based development platforms provided by Intrinsyc
Multiple form factors supported
− Exposed boards (DragonBoards)
− Tablets (MDPs)
Hardware support provided by Intrinsyc
− http://www.intrinsyc.com/products/qualcomm/dragonboard-development-kits.aspx
Notable features: Hardware development platforms

17
C and Assembly optimized libraries for common operations
− FIR, IIR
− FFT, IFFT, FHT
− Up sampler
− Computer vision (FastCV)
− feature/object detection
− motion & object tracking
− color conversion
− 3D reconstruction
− image processing
− Lots more …
Notable features: Optimized Hexagon libraries

18
QuRT™ software: real-time operating system for the Hexagon processor
Low overhead both in memory and processing requirements
Supports
− Priority-based preemptive multithreading
− Thread synchronization and communication (mutex, signal, semaphore, barriers, pipes, …)
− Timers
− Interrupts
− Exceptions
− Memory management
Notable features: RTOS
QuRT is a product of Qualcomm Technologies, Inc.

19
ISO C/C++ compiler
Assembler
Standard (libc) and processor specific libraries
Intrinsics
Linker and archiver
Debugger (gdb)
Profiler (gprof)
Misc utilities
− ELF viewer
− nm
− strings
− size
Notable features: Comprehensive tool chain

20
Runtime diagnostic logging
Viewable in logcat or on host (PC)
Integrated with the Hexagon SDK Eclipse plugin
Supported in simulation and on-target
Notable features: Diagnostic logging

21
What can you do with
the Hexagon SDK?

22
Offload compute tasks from the application processor onto the Hexagon DSP
− Lower power
− Frees up resources on the application processor
− Appears to caller as if calling a local library
Easier to do
− Define interface in IDL
− Use existing C code or optimize with Hexagon DSP assembly
− Build using the Hexagon Tools suite
− Call via the stub library on the application processor.
Moving data between processors is transparent to callers and implementers
− Cache coherent
− Zero copy
Calculator example in the Hexagon SDK
General compute
What can you do with the Hexagon SDK?

23
Add custom algorithms, encoders, and decoders to the audio framework
Dynamically load and unload audio use cases on an as-needed basis
− Saves memory
Communicate with your audio module at runtime
− Tuning
− Feature enabling/disabling
− User controlled settings
Clearly defined interface to implement (APPI or CAPI)
− Unit test frameworks to validate that your module is ready to run on the Hexagon DSP
Lots of APPI and CAPI examples in the Hexagon SDK
Audio

24
Add custom processing algorithms to the voice framework
Communicate with your voice module at runtime
− tuning
− Feature enabling/disabling
− User controlled settings
Clearly defined interface to implement (CAPI_V2)
− Unit test frameworks to validate that your module is ready to run on the Hexagon DSP
CAPI_V2 passthru and gain examples in the Hexagon SDK
Voice

25
Library of frequently used functions optimized to run on mobile devices
− Math / vector operations
− Image processing and transformation
− Feature and object detection
− 3D reconstruction
− Color conversion
− Clustering and search
− Motion and object tracking
− Share and drawing
− Memory management
downscaleBy2 and cornerapp examples in the Hexagon SDK
Computer vision

27
developer.qualcomm.com
Augmented
reality
Context
awareness
Peer
to peer
Wireless
health
Graphics
& gaming
Computer
vision
Android
applications
App development SDKs & tools
Marketing opportunities
Case Studies & Tutorials
News & Information
Hardware development platforms
Forums & technical support
Tools & resources to help developers build, integrate, and optimize
Qualcomm Developer Network

28
For more information on Qualcomm, visit us at:
www.qualcomm.com & www.qualcomm.com/blog
Thank you
FOLLOW US ON:
© 2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All rights reserved.
Qualcomm, DragonBoard, and Hexagon are trademarks of Qualcomm Incorporated, registered in the United States and other countries. Uplinq and QuRT are trademarks of Qualcomm Incorporated. All trademarks of
Qualcomm Incorporated are used with permission. Other product and brand names may be trademarks or registered trademarks of their respective owners.
References in this presentation to “Qualcomm” may mean Qualcomm Incorporated, Qualcomm Technologies, Inc., and/or other subsidiaries or business units within the Qualcomm corporate structure, as applicable.
Qualcomm Incorporated includes Qualcomm’s licensing business, QTL, and the vast majority of its patent portfolio. Qualcomm Technologies, Inc., a wholly-owned subsidiary of Qualcomm Incorporated, operates, along with
its subsidiaries, substantially all of Qualcomm’s engineering, research and development functions, and substantially all of its product and services businesses, including its semiconductor business, QCT.

29
Additional Slides /
APPENDIX

30
Notable features: Eclipse IDE plugin: On-target debug
Source: Hexagon SDK

31
Notable features: Eclipse IDE plugin: Compute off-load
Source: Hexagon SDK

32
Notable features: Eclipse IDE plugin: Custom views
AMDB View
Identifies the devices
(including hot plug),
provides the facility to
add/remove/configure
the audio modules
Quick Profile Viewer
Displays static/dynamic
profiling information for a
project,
Device Viewer
Displays the connected
devices and their state,
Source: Hexagon SDK

Optimize multimedia with Qualcomm Hexagon SDK

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Optimize multimedia with Qualcomm Hexagon SDK

Similar to Optimize multimedia with Qualcomm Hexagon SDK (20)

More from Qualcomm Developer Network

More from Qualcomm Developer Network (20)

Recently uploaded

Recently uploaded (20)

Optimize multimedia with Qualcomm Hexagon SDK