This document introduces MPEG-M (MPEG Extensible Middleware), which defines a set of APIs that provide simple access to complex MPEG technologies. MPEG-M aims to accelerate media application development by hiding the complexity of MPEG standards behind these APIs. It establishes a common middleware layer that allows applications to easily use various MPEG engines without needing deep knowledge of the standards. The MPEG-M standard also enables applications and engines to be developed and updated independently by different parties.

1. http://mxm.wg11.sc29.org/ mxm@lists.uni-klu.ac.at http://wg11.sc29.org/mxmsvn/repos Accelerating Media Business Developments MPEG-M: MPEG Extensible Middleware Christian Timmerer Klagenfurt University (UNI-KLU)  Faculty of Technical Sciences (TEWI) Department of Information Technology (ITEC)  Multimedia Communication (MMC) http://research.timmerer.com  http://blog.timmerer.com  mailto:christian.timmerer@itec.uni-klu.ac.at Acknowledgments L. Chiariglione, F. Chiariglione M. Preda, M. Eberhard, I. Arsov, A. Defino Workshop on New Multimedia Technologies and ApplicationsXidian University, China October 31, 2009

2. What if … … one does not need to have in-depth knowledge of specific MPEG technologies / standards? … one is able to start with application/business development as soon as some (reference) software becomes available? … one is able to exchange applications’ underlying (reference) software with optimized one at no cost? 2009/10/31 Christian Timmerer, Klagenfurt University, Austria 2

3. 2009/10/31 Christian Timmerer, Klagenfurt University, Austria 3

4. Outline Introduction – Overview Architecture Application Programming Interface (API) Example Instantiations Fully Interoperable Streaming [Including MPEG-4 3D Graphics Sharing Protected Contents] [Conclusions – References] 2009/10/31 Christian Timmerer, Klagenfurt University, Austria 4

5. Introduction – Overview A set of APIs to access possibly all MPEG technologies Simple methods to call complex functionalities inside MXM engines “Thin” applications because the complexity is in the MXM engines Replacement of MXM engines with better performing ones at no cost Creation of a global market of MXM Engines, MXM Applications and MXM Devices 2009/10/31 Christian Timmerer, Klagenfurt University, Austria 5

6. Introduction – Overview (cont’d) The MXM standard – ISO/IEC 23006 – is subdivided in four parts: Part 1 - MXM Architecture and Technologies specifies the MXM architecture and references the technologies that are part of an MXM implementation; Part 2 - MXM Application Programming Interfaces (APIs): specifies the MXM APIs; Part 3 - MXM Conformance and Reference Software: specifies conformance tests and the software implementation of the standard Part 4 – MXM Protocols: specifies a set of protocols enabling distributed applications to exchange information related to content items and parts thereof, including rights and protection information 2009/10/31 Christian Timmerer, Klagenfurt University, Austria 6

7. MXM Application 1 MXM Application 2 MXM Engine APIs‏ MXM Device MPEG21 File Engine Digital Item Engine REL Engine IPMP Engine Other Engines Security Engine Scene Engine Content Metadata Engine Media Framework Engine OS Drivers, Accelerators, Controllers, etc. HW 7 2009/10/31 Christian Timmerer, Klagenfurt University, Austria

8. MXM Application 3 MXM Application 1 MXM Application 2 MXM Orchestrator API‏ MXM Device MPEG21 File Engine Digital Item Engine REL Engine IPMP Engine Orchestrator Engine Other Engines Security Engine Scene Engine Content Metadata Engine Media Framework Engine OS Drivers, Accelerators, Controllers, etc. HW 8 2009/10/31 Christian Timmerer, Klagenfurt University, Austria

9. MXM Protocols License Provider Device Content Identific. Device Content Creation Device End-User Device Content Provider Device DRM Tool Provider Device 9 2009/10/31 Christian Timmerer, Klagenfurt University, Austria

10. MXM Application MXM Application MXM OS MXM OS Computing Platform Computing Platform Enabling MXM Apps to communicate 10 2009/10/31 Christian Timmerer, Klagenfurt University, Austria

11. An Impressive Number of Standards… MPEG-7 Visual Advanced Audio Coding LASeR MDS Event Reporting Reconfigurable Video Coding BIFS HE AAC IPMPX Digital Item Processing Digital Item Declaration File Format Audio Lossless Coding Intellectual Property Management and Protection Media Value Chain Ontology Rights Expression Language Digital Item Identification XML IPMP messages Digital Item Streaming IPMP Components Digital Item Adaptation MPQF 11 2009/10/31 Christian Timmerer, Klagenfurt University, Austria

12. Motivation of providing API MPEG specifications: huge amount of technology 157 standards in ISO/IEC 14496 (MPEG-4) family 43 standards in ISO/IEC 15938 (MPEG-7) family 33 standards in ISO/IEC 21000 (MPEG-21) family … Around 11 000 pages, 1,5 m high when printed However, majority of people developing MPEG related applications do not need to know that is inside the boxes, but only how to use it 2009/10/31 12 Christian Timmerer, Klagenfurt University, Austria

13. The MXM Approach 1. Creating wrapping libraries, called engines 2. Opening the engines at ESSENTIAL points only 3. Documenting the IN/OUT points in another … MPEG standard: 23006-2 MXM API - only 37 pages , - around 500 methods 2009/10/31 13 Christian Timmerer, Klagenfurt University, Austria

14. List of MXM Engines ContentProtocolEngine ContentSearchEngine DIAEngine DIDEngine DISEngine DomainEngine EREngine IPMPEngine IPMPToolProtocolEngine LicenseProtocolEngine MediaFrameworkEngine MetadataEngine MPEG21FileEngine MVCOEngine OrchestratorEngine RELEngine RenderingEngine SecurityEngine 14 2009/10/31 Christian Timmerer, Klagenfurt University, Austria

15. MXM APIs of Each Engine have been Divided into… Creation e.g. Encode a raw audio track Create an MPEG-7 metadata description Access e.g. Get data from a Digital Item Decode a video Editing e.g. Add an elementary stream to a multiplexed content Engine-specific e.g. RELEngine: authorise(license, query) LicenseProtocolEngine: requestLicense(licenseID, serviceURL) 15 2009/10/31 Christian Timmerer, Klagenfurt University, Austria

16. Fully Interoperable Streaming ofMedia Resources in Heterogeneous Environments 2009/10/31 Christian Timmerer, Klagenfurt University, Austria 16

17. Join the MXM Development Team! Why you should join Plenty of interesting work still has to be done Enough space for student projects, master thesis, PhD thesis, etc. You can choose between Java, C++ or start a new implementation in another language! It gives you visibility on a broad set of MPEG technologies Web site, blog, reflector, and soon more utilities Friendly and collaborative environment  17 2009/10/31 Christian Timmerer, Klagenfurt University, Austria

18. Thank you for your attention! ... questions, comments, etc. are welcome … Ass.-Prof. Dipl.-Ing. Dr. Christian Timmerer Klagenfurt University, Department of Information Technology (ITEC) Universitätsstrasse 65-67, A-9020 Klagenfurt, AUSTRIA christian.timmerer@itec.uni-klu.ac.at http://research.timmerer.com/ Tel: +43/463/2700 3621 Fax: +43/463/2700 3699 © Copyright: Christian Timmerer 18 2009/10/31 Christian Timmerer, Klagenfurt University, Austria http://mxm.wg11.sc29.org/ mxm@lists.uni-klu.ac.at http://wg11.sc29.org/mxmsvn/repos