OpenMAX is a series of interfaces that allow for better integration of media components. It includes three levels of interfaces: OpenMAX IL for connecting media components, OpenMAX AL for controlling media playback, and OpenMAX DL for defining basic media primitives. OpenMAX enables portable and flexible development of media applications across different hardware architectures.

1. OpenMAX Overview

2. OpenMAX סדרה של ממשקים המאפשרים אינטגרציה טוב יותר של רכיבי מדיה . לדוגמא : דוחסי וידאו ואודיו מוקסרים פרסרים שימושי במיוחד לאינטגרציית CODEC באנדרואיד

3. ממשקי ה - OpenMAX ל - OpenMAX שלוש רמות של ממשקים : OpenMAX IL – ממשק ברמה של directshow המאפשר לחבר קומפונטטות מדיה וליצור גרפים OpenMAX AL – ממשק לרמת נגן המדיה המאפשר לשלוט בנגן ברמה של Pause, Play, Stop OpenMAX DL - ממשק ברמת ה - Primitives , פונקציות בסיסיות כמו DCT , FFT , FIR וכו

4. OpenMAX – Streaming Media Media Engines - CPUs, DSP, Hardware Accelerators “ Development Layer” Defines media primitives and concurrency constructs Audio Components e.g. MP3 Media components can be integrated into flexible media graphs for advanced streaming media processing Media components can be written using primitives for portability across diverse parallel and serial silicon architectures Application Platform Media Framework “ Application Layer” Defines high-level playback and recording interface API Media applications can be written portably, independent of the underlying media platform IL DL AL “ Integration Layer” Defines media component interfaces Video Components e.g. H.264 Image Components e.g. JPEG OpenMAX layers can be implemented together or independently from the other layers

5. OpenMAX DL – Component Portability OpenMAX DL is a library of key static primitive functions Designed to cover 80% of the processing required in a multimedia codec Abstracts the ISA (Instruction Set Architecture) from the multimedia codec Enables faster codec development time and faster porting of existing codecs Enables third party codec vendors to sell processor-agnostic codecs Multi-core architectures (i.e. ARM + DSP) gain greater code reuse between cores

6. OpenMAX DL – Component Portability Combinatorial Problem A wide range of media acceleration silicon using many diverse architectures Silicon vendors supply optimized OpenMAX DL library for rapid porting of codecs across multiple accelerators An increasing number of multimedia API codecs for video, audio, graphics and images

7. OpenMAX DL Domains Video Domain MPEG-4 SP/H.263 BL (encode and decode) H.264 (encode and decode) Image Codec Domain JPEG (encode and decode) Image Processing Domain Color space conversion Pixel packing/unpacking De-blocking / de-ringing Rotation, scaling, compositing, etc. Multimedia Audio Domain MP3 AAC Signal Processing Domain FIR IIR FFT Dot Product

8. OpenMAX IL – Component Networks Enables arbitrary multimedia pipelines by plugging blocks together Abstracts multimedia functionality with block interfaces Wide variety of building blocks for imaging, video and audio functions Encode, decode, apply an effect, capture, render, split, mix, etc Enables blocks from different sources to work together Blocks can be implemented in software or hardware Imaging Renderers Voice Audio CODEC CODEC Video

9. OpenMAX IL “Component” A component is a building block encapsulating one function Plumbing Each component port is the entry point or exit point for a stream of data Ports are connected between components Knobs A component parameter is a value that is set prior to component execution A component config is a value that may be set during component execution Examples: rate, volume, resolution, scaling, bit rate

10. OpenMAX IL “Component” Control – a standard interface common to all components to control: The connection of ports and how data flows in and out of ports Query/set configs and parameters State management

11. OpenMAX IL Example Graph Standardized component interfaces enable flexible media graphs Includes multi-stream synchronization Allows for custom plug-ins *.mp4 / *.3gp File Reader Clock for AV Sync Audio Decoder Audio Renderer Video Scheduler Video Renderer Speakers Display AAC Audio MPEG4/ H.264 Video Time Data Video Decoder Decompressed Video OpenMAX Component Interfaces Enables components to be flexibly connected in any graph topology Example: MPEG-4 video synchronized with AAC audio decode Plug-in E.g. send dance beats to 3D application for visualization

12. OpenMAX IL 1.1 Solution: OpenMAX IL 1.1 defines a set of standard components To make most streaming applications easier Standardized interfaces and controls Readers/writers: 3gp, asf, image, video, audio Audio decoders/encoders: AAC, AMR, MP3, WMA, Real Video Audio post-processor: stereo widening, equalizer, reverb Video decoders/encoders: MPEG4, H.264, etc Image decoders/encoders: JPEG …

13. OpenMAX IL 1.2 Solution: OpenMAX 1.2 defines two sets of functionality “ Playback Enabled” - Defines a group of standard components sufficient for full featured playback of audio/video/image “ Recording Enabled” - Defines a group of standard components sufficient for full featured capture of audio/video/image Mirrors OpenMAX AL profiles

14. OpenMAX AL – Component Control OpenMAX IL is powerful but complex Most application developers just want to playback and record media Specifying where the content comes from Specifying where the content should be rendered to Manipulate a few playback controls Have simple configurability That’s what OpenMAX AL provides… A simple high-level multimedia API for playback and recording use cases

15. OpenMAX AL – Obj. Oriented Media OpenMAX AL - simplified object-oriented streaming media Common use cases – but also extensible Can be implemented over OpenMAX IL OpenMAX AL Media Objects enable PLAY and RECORD of media Media Objects input and output to Devices Perform some operation on an input and emit the result as output Can handle audio, images, video with synchronized audio

16. OpenMAX AL – Obj. Oriented Media Objects have control interfaces Play, Seek, Rate, Audio, Display Region, Metadata Extraction Record, Camera, Video Encoder, Audio Encoder, Metadata Insertion, Radio, MIDI

17. OpenMAX AL Video Playback Example Create Engine object (Graph) To drive this session Create Audio Output Mix object Method on Engine interface, Mix object drives audio output devices Create Media Player object Method on Engine interface Input is URI pointing to a local 3gp file Output drives display and audio output mix

18. OpenMAX AL Video Playback Example Register event callback Method on Media Player interface Set PlayState to Playing Method on Media Player interface Wait for end of file event Via registered callback

19. Advanced OpenMAX AL Features Extensive camera controls Analog radio controls Audio routing Metadata extraction and insertion