OpenMAX Overview


Published on

OpenMAX Overview

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • WMP Overview 03/01/11 Copyright 2005, All rights reserved
  • WMP Overview 03/01/11 Copyright 2005, All rights reserved
  • WMP Overview 03/01/11 Copyright 2005, All rights reserved
  • WMP Overview 03/01/11 Copyright 2005, All rights reserved
  • WMP Overview 03/01/11 Copyright 2005, All rights reserved
  • OpenMAX Overview

    1. 1. OpenMAX Overview
    2. 2. OpenMAX <ul><li>סדרה של ממשקים המאפשרים אינטגרציה טוב יותר של רכיבי מדיה . לדוגמא : </li></ul><ul><li>דוחסי וידאו ואודיו </li></ul><ul><li>מוקסרים </li></ul><ul><li>פרסרים </li></ul><ul><li>שימושי במיוחד לאינטגרציית CODEC באנדרואיד </li></ul>
    3. 3. ממשקי ה - OpenMAX <ul><li>ל - OpenMAX שלוש רמות של ממשקים : </li></ul><ul><li>OpenMAX IL – ממשק ברמה של directshow המאפשר לחבר קומפונטטות מדיה וליצור גרפים </li></ul><ul><li>OpenMAX AL – ממשק לרמת נגן המדיה המאפשר לשלוט בנגן ברמה של Pause, Play, Stop </li></ul><ul><li>OpenMAX DL - ממשק ברמת ה - Primitives , פונקציות בסיסיות כמו DCT , FFT , FIR וכו </li></ul>
    4. 4. OpenMAX – Streaming Media Media Engines - CPUs, DSP, Hardware Accelerators “ Development Layer” Defines media primitives and concurrency constructs Audio Components e.g. MP3 Media components can be integrated into flexible media graphs for advanced streaming media processing Media components can be written using primitives for portability across diverse parallel and serial silicon architectures Application Platform Media Framework “ Application Layer” Defines high-level playback and recording interface API Media applications can be written portably, independent of the underlying media platform IL DL AL “ Integration Layer” Defines media component interfaces Video Components e.g. H.264 Image Components e.g. JPEG OpenMAX layers can be implemented together or independently from the other layers
    5. 5. OpenMAX DL – Component Portability <ul><li>OpenMAX DL is a library of key static primitive functions </li></ul><ul><ul><li>Designed to cover 80% of the processing required in a multimedia codec </li></ul></ul><ul><li>Abstracts the ISA (Instruction Set Architecture) from the multimedia codec </li></ul><ul><ul><li>Enables faster codec development time and faster porting of existing codecs </li></ul></ul><ul><li>Enables third party codec vendors to sell processor-agnostic codecs </li></ul><ul><ul><li>Multi-core architectures (i.e. ARM + DSP) gain greater code reuse between cores </li></ul></ul>
    6. 6. OpenMAX DL – Component Portability Combinatorial Problem A wide range of media acceleration silicon using many diverse architectures Silicon vendors supply optimized OpenMAX DL library for rapid porting of codecs across multiple accelerators An increasing number of multimedia API codecs for video, audio, graphics and images
    7. 7. OpenMAX DL Domains <ul><li>Video Domain </li></ul><ul><ul><li>MPEG-4 SP/H.263 BL (encode and decode) </li></ul></ul><ul><ul><li>H.264 (encode and decode) </li></ul></ul><ul><li>Image Codec Domain </li></ul><ul><ul><li>JPEG (encode and decode) </li></ul></ul><ul><li>Image Processing Domain </li></ul><ul><ul><li>Color space conversion </li></ul></ul><ul><ul><li>Pixel packing/unpacking </li></ul></ul><ul><ul><li>De-blocking / de-ringing </li></ul></ul><ul><ul><li>Rotation, scaling, compositing, etc. </li></ul></ul><ul><li>Multimedia Audio Domain </li></ul><ul><ul><li>MP3 </li></ul></ul><ul><ul><li>AAC </li></ul></ul><ul><li>Signal Processing Domain </li></ul><ul><ul><li>FIR </li></ul></ul><ul><ul><li>IIR </li></ul></ul><ul><ul><li>FFT </li></ul></ul><ul><ul><li>Dot Product </li></ul></ul>
    8. 8. OpenMAX IL – Component Networks <ul><li>Enables arbitrary multimedia pipelines by plugging blocks together </li></ul><ul><ul><li>Abstracts multimedia functionality with block interfaces Wide variety of building blocks for imaging, video and audio functions </li></ul></ul><ul><ul><li>Encode, decode, apply an effect, capture, render, split, mix, etc </li></ul></ul><ul><li>Enables blocks from different sources to work together </li></ul><ul><ul><li>Blocks can be implemented in software or hardware </li></ul></ul>Imaging Renderers Voice Audio CODEC CODEC Video
    9. 9. OpenMAX IL “Component” <ul><li>A component is a building block encapsulating one function </li></ul><ul><li>Plumbing </li></ul><ul><ul><li>Each component port is the entry point or exit point for a stream of data </li></ul></ul><ul><ul><li>Ports are connected between components </li></ul></ul><ul><li>Knobs </li></ul><ul><ul><li>A component parameter is a value that is set prior to component execution </li></ul></ul><ul><ul><li>A component config is a value that may be set during component execution </li></ul></ul><ul><ul><li>Examples: rate, volume, resolution, scaling, bit rate </li></ul></ul>
    10. 10. OpenMAX IL “Component” <ul><li>Control – a standard interface common to all components to control: </li></ul><ul><ul><li>The connection of ports and how data flows in and out of ports </li></ul></ul><ul><ul><li>Query/set configs and parameters </li></ul></ul><ul><ul><li>State management </li></ul></ul>
    11. 11. OpenMAX IL Example Graph <ul><li>Standardized component interfaces enable flexible media graphs </li></ul><ul><li>Includes multi-stream synchronization </li></ul><ul><li>Allows for custom plug-ins </li></ul>*.mp4 / *.3gp File Reader Clock for AV Sync Audio Decoder Audio Renderer Video Scheduler Video Renderer Speakers Display AAC Audio MPEG4/ H.264 Video Time Data Video Decoder Decompressed Video OpenMAX Component Interfaces Enables components to be flexibly connected in any graph topology Example: MPEG-4 video synchronized with AAC audio decode Plug-in E.g. send dance beats to 3D application for visualization
    12. 12. OpenMAX IL 1.1 <ul><li>Solution: OpenMAX IL 1.1 defines a set of standard components </li></ul><ul><ul><li>To make most streaming applications easier </li></ul></ul><ul><li>Standardized interfaces and controls </li></ul><ul><ul><li>Readers/writers: 3gp, asf, image, video, audio </li></ul></ul><ul><ul><li>Audio decoders/encoders: AAC, AMR, MP3, WMA, Real Video </li></ul></ul><ul><ul><li>Audio post-processor: stereo widening, equalizer, reverb </li></ul></ul><ul><ul><li>Video decoders/encoders: MPEG4, H.264, etc </li></ul></ul><ul><ul><li>Image decoders/encoders: JPEG … </li></ul></ul>
    13. 13. OpenMAX IL 1.2 <ul><li>Solution: OpenMAX 1.2 defines two sets of functionality </li></ul><ul><ul><li>“ Playback Enabled” - Defines a group of standard components sufficient for full featured playback of audio/video/image </li></ul></ul><ul><ul><li>“ Recording Enabled” - Defines a group of standard components sufficient for full featured capture of audio/video/image </li></ul></ul><ul><li>Mirrors OpenMAX AL profiles </li></ul>
    14. 14. OpenMAX AL – Component Control <ul><li>OpenMAX IL is powerful but complex </li></ul><ul><li>Most application developers just want to playback and record media </li></ul><ul><ul><li>Specifying where the content comes from </li></ul></ul><ul><ul><li>Specifying where the content should be rendered to </li></ul></ul><ul><ul><li>Manipulate a few playback controls </li></ul></ul><ul><ul><li>Have simple configurability </li></ul></ul><ul><li>That’s what OpenMAX AL provides… </li></ul><ul><ul><li>A simple high-level multimedia API for playback and recording use cases </li></ul></ul>
    15. 15. OpenMAX AL – Obj. Oriented Media <ul><li>OpenMAX AL - simplified object-oriented streaming media </li></ul><ul><ul><li>Common use cases – but also extensible </li></ul></ul><ul><ul><li>Can be implemented over OpenMAX IL </li></ul></ul><ul><li>OpenMAX AL Media Objects enable PLAY and RECORD of media </li></ul><ul><ul><li>Media Objects input and output to Devices </li></ul></ul><ul><ul><li>Perform some operation on an input and emit the result as output </li></ul></ul><ul><ul><li>Can handle audio, images, video with synchronized audio </li></ul></ul>
    16. 16. OpenMAX AL – Obj. Oriented Media <ul><li>Objects have control interfaces </li></ul><ul><ul><li>Play, Seek, Rate, Audio, Display Region, Metadata Extraction </li></ul></ul><ul><ul><li>Record, Camera, Video Encoder, Audio Encoder, Metadata Insertion, Radio, MIDI </li></ul></ul>
    17. 17. OpenMAX AL Video Playback Example <ul><li>Create Engine object (Graph) </li></ul><ul><ul><li>To drive this session </li></ul></ul><ul><li>Create Audio Output Mix object </li></ul><ul><ul><li>Method on Engine interface, Mix object drives audio output devices </li></ul></ul><ul><li>Create Media Player object </li></ul><ul><ul><li>Method on Engine interface </li></ul></ul><ul><ul><li>Input is URI pointing to a local 3gp file </li></ul></ul><ul><ul><li>Output drives display and audio output mix </li></ul></ul>
    18. 18. OpenMAX AL Video Playback Example <ul><li>Register event callback </li></ul><ul><ul><li>Method on Media Player interface </li></ul></ul><ul><li>Set PlayState to Playing </li></ul><ul><ul><li>Method on Media Player interface </li></ul></ul><ul><li>Wait for end of file event </li></ul><ul><ul><li>Via registered callback </li></ul></ul>
    19. 19. Advanced OpenMAX AL Features <ul><li>Extensive camera controls </li></ul><ul><li>Analog radio controls </li></ul><ul><li>Audio routing </li></ul><ul><li>Metadata extraction and insertion </li></ul>