Uplin q'14 harikishan desineni - enhance qualcomm® snapdragon™ audio using android audio ap_is

1. 1

2. Enhance Qualcomm®
Snapdragon™ audio using
Android Audio APIs
Harikishan Desineni,
Director of Engineering
Qualcomm Innovation Center, Inc.
Qualcomm Snapdragon is a product of Qualcomm Technologies, Inc.

3. 3
Agenda
Session
overview
1
How Qualcomm
Technologies
incorporated the
digital signal
processor (DSP)
2
Snapdragon Audio
SDK
3
Open DSP program
4
Demo
5

4. 4
Includes the following sound effects:
− Equalizer (EQ) – Adjusts the sound level of certain frequency ranges by use of audio
filters to improve the quality of overall perceived sounds
− Virtualizer – Virtualizes audio channels to compensate for the mismatch between the
loudspeaker setup used when producing the audio, and the audio output device used
when the audio is played back by the end user
− Reverb – Simulates the reverberation that results from reflections from surrounding
walls and objects in an environment to make the sound more live and real
− BassBoost – Dynamically boosts low-frequency content of an audio signal to achieve
better listening experience on bass for a headset device
How Qualcomm® Snapdragon™ Audio+ uses the DSP
Qualcomm Snapdragon Audio+ is a product of Qualcomm Technologies, Inc.

5. 5
Media playback block diagram
Surface Flinger Audio Flinger
MDP/MDDI
Media Player
Media Player Service
StageFright
User space
Java
User space
Native
Kernel
space
DSP
Video OMX
Component
Audio OMX
Component
Video HW
PCM-audio
Audio HAL
Audio HW
Media Server
Audio DriversDisplay Drivers Video Drivers

6. 6
Audio framework: effects offloaded to DSP
Media Player
Media Player
Service
Awesome
Player
Audio Track AudioFlinger
Qualcomm®
Hexagon™
AudioEffects
Software
AudioEffect
BassBosst
Virtualizer
ReverB
X
Qualcomm Hexagon is a product of Qualcomm Technologies, Inc.

7. 7
Snapdragon Audio SDK

8. 8
Snapdragon Audio SDK
Fluence™ Pro EC NS detection
Snapdragon SDK for Android: Device Info
The Device Info feature set is designed to allow an application to detect if the device it is running on has a
Snapdragon processor. This insight lets you make informed decisions about how to take full advantage of
features enabled by Snapdragon processors.
Using Device Info is designed to let you tap into the following hardware features to help improve the
performance of your applications:
− Is Snapdragon
This feature is designed to allow the application to detect at runtime if the device has a Snapdragon processor
− Echo Cancellation/Noise Suppression
Echo Cancellation/Noise Suppression is designed to foster better audio communication quality at lower power by
allowing full duplex mode during VOIP calls. These features are specifically tuned to each device, and use technology
available on normal mobile voice calls so you can offload CPU/software-based echo cancellation and noise suppression.
Fluence Pro is a product of Qualcomm Technologies, Inc.

9. 9
Snapdragon Audio SDK
com.qualcomm.snapdragon.sdk.deviceinfo
Class Description
DeviceInfo The DeviceInfo API provides a mechanism to query a device about the presence of certain functionalities.
Class Description
DeviceInfo.Features The list of features that can be detected with the function isFeatureSupported(Feature, Context)
Sample code is subject to SDK license agreement. Source: http://developer.qualcomm.com

10. 10
Snapdragon Audio SDK
public static final DeviceInfo.Features FEATURE_FLUENCE
Noise Cancellation Technology
− If isFeatureSupported returns true for this feature, then it can be assumed that echo
cancellation is supported on the voice path.
− If isFeatureSupported returns true for this feature, and the standard Android API
android.media.audiofx.AcousticEchoCanceler.isAvailable() also returns true, then it can
be assumed that echo cancellation is also available on the audio path.

11. 11
Snapdragon Audio SDK
FEATURE_SURROUND_SOUND_RECORDING
public static final DeviceInfo.Features FEATURE_SURROUND_SOUND_RECORDINGSurround
Sound Recording is designed to allow for 5.1 surround sound input using existing
microphones on the device

12. 12
OpenDSP program

13. 13
Audio Modules – Common Interface
Common interface for audio
modules that are defined to ease
integration with the Elite
Framework
Audio Postprocessor Interface
(APPI) for audio processing
algorithms
Common Audio Processor Interface
(CAPI) for audio codec modules
Processing
Algorithm
Wrapper with
common interface (APPI)
Codec
Decoder / Encoder
Wrapper with
common interface (CAPI)
Note: The terms algorithm, module, and library are used interchangeably in this presentation.

14. 14
Audio Modules – Static and Dynamic
Two types of modules, based on loading:
− Static modules (*.lib)
− Linked to the aDSP image at compile time
− Symbol are resolved at compile time
− aDSP image must be recompiled when a module changes
− Dynamic modules (*.so)
− Loaded at run-time into aDSP heap memory
− Symbols are resolved when the module is loaded into memory
− aDSP image does not need to be recompiled when a
module changes
Each module is identified by unique global module ID
Audio static modules are added to the AMDB at boot time
Audio dynamic modules are added to the AMDB at run time
APPI mod-n (*.so)
CAPI mod-3 (*.so)
CAPI mod-2 (*.lib)
APPI mod-1 (*.lib)

15. 15
Per Object PostProcessing (POPP)
Per Object PreProcessing (POPreP )
Processing chain per audio stream
Default POPP topologies:
− Playback session
− Record session
DSP audio
mixer
Downmix
if needed
Resampler Audio EncoderVolume
DSP audio
mixer
Volume Audio DecoderResampler
POPP Topologies

16. 16
Common Object PostProcessing (COPP)
Common Object PreProcessing (COPreP)
Processing chain per device
Default COPP topologies:
− Playback session
− Record session
AFE
Audio
pregain
AFE CodecRX
Gain
DSP Audio
Mixer
Audio
Volume
DSP Audio
Mixer
COPP Topologies

17. 17
Custom Audio Topology – Static and Dynamic
Two types of audio topologies:
− Static topologies
− Created at compile time
− aDSP image must be recompiled if the topology changes
− Dynamic topology
− Created at run time
− aDSP image does not need to be recompiled if the topology changes
− APIs are defined for adding custom dynamic topologies from the applications processor:
Custom topology is designed to use any module defined in the AMDB by using Module IDs
Customized topology with an OEM algorithm
ASM_CMD_ADD_TOPOLOGIES // for POPP and POPREP
ADM_CMD_ADD_TOPOLOGIES_V5 // for COPP and COPREP
Audio Mixer Volume
OEM
Algorithm
Audio DecoderResampler

18. 18
Audio Dynamic Topology
How are dynamic topologies added at run time?
APR
ADM
Topo DB
ASM_CMD_ADD_TOPOLOGIES T2 (-M1-M4-), T3 (-M1-M2)
topo_db_add_topologies()
T1
T2
T3
aDSP
AMDB
M1
M2
M3
M4
New audio sessions can be created using T2 and T3
topologies after adding them to the topology databaseClient Processor
ASM

19. 19
ISV Module1 ISV Module2
Application
HAL
RTAC Kernel Driver
APR Handler
DSP
Payload
ASM_CMD_SET_PP_PARAMS/
AMD_CMD_SET_PP_PARAMS
RTAC Userspace module
Send_rtac()
ACDB Loader
Acdb_store()
Acdb_send()
Acdb_get()
acdb file
Get/setParameter(key”,value)
Software Architecture – Block Diagram

20. 20
Software Architecture – Android APIs
Existing set parameter android API extended to have new key value pairs
Setparameter() with persistence
− set_parameter(key=”ISV_PERSIST_CFG”, value=” DataBinary, *”);
− Data Binary – Entire calibration data binary that is stored in calibration database. Even if user changes single
parameter through UI, the app is responsible for packaging the entire data binary for persistence
Setparameter() without persistence
− set_parameter(key=”ISV_NOPERSIST_CFG”, value, “List_MID_PID, *”);
− List_MID_PID – List of dynamically modified calibration to be applied on ISV modules in ADSP. This could be an
array of Module ID, Parameter ID and data, or could be only one set of ModuleId, Parameter ID and values. Here, the
data is not persisted in audio calibration database. The app is responsible for packaging the modified modules and
parameters.

21. 21
Demo

22. 22
developer.qualcomm.com
Augmented
reality
Context
awareness
Peer
to peer
Wireless
health
Graphics
& gaming
Computer
vision
Android
applications
App development SDKs & tools
Marketing opportunities
Case Studies & Tutorials
News & Information
Hardware development platforms
Forums & technical support
Tools & resources to help developers build, integrate, and optimize
Qualcomm Developer Network

23. 23
For more information on Qualcomm, visit us at:
www.qualcomm.com & www.qualcomm.com/blog
Thank you
FOLLOW US ON:
© 2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All rights reserved.
Qualcomm, Snapdragon, Hexagon and Fluence are trademarks of Qualcomm Incorporated, registered in the United States and other countries. Uplinq is a trademark of Qualcomm Incorporated. All trademarks of
Qualcomm Incorporated are used with permission. Other product and brand names may be trademarks or registered trademarks of their respective owners.
References in this presentation to “Qualcomm” may mean Qualcomm Incorporated, Qualcomm Technologies, Inc., and/or other subsidiaries or business units within the Qualcomm corporate structure, as applicable.
Qualcomm Incorporated includes Qualcomm’s licensing business, QTL, and the vast majority of its patent portfolio. Qualcomm Technologies, Inc., a wholly-owned subsidiary of Qualcomm Incorporated, operates, along with
its subsidiaries, substantially all of Qualcomm’s engineering, research and development functions, and substantially all of its product and services businesses, including its semiconductor business, QCT.