This document introduces MPEG-M (MPEG Extensible Middleware), which provides a standardized set of APIs to access various MPEG technologies through "engines". The MXM APIs simplify application development by hiding the complexity of MPEG standards. The MXM architecture includes engines for MPEG-21, MPEG-4, IPMP, and other technologies. Applications can access these engines through simple MXM API calls to perform tasks like streaming media, including 3D graphics, and sharing protected content. The goal of MXM is to accelerate media business by providing a common platform and set of protocols for distributed multimedia applications and services.

1. http://mxm.wg11.sc29.org/mxm@lists.uni-klu.ac.athttp://wg11.sc29.org/mxmsvn/reposAccelerating Media Business DevelopmentsMPEG-M: MPEG Extensible MiddlewareChristian Timmerer, FilippoChiariglione, Marius PredaKlagenfurt University (UNI-KLU)  Faculty of Technical Sciences (TEWI)Department of Information Technology (ITEC)  Multimedia Communication (MMC)http://research.timmerer.com  http://blog.timmerer.com  mailto:christian.timmerer@itec.uni-klu.ac.atAcknowledgmentsL. Chiariglione, M. Eberhard, I. Arsov, A. Difino

2. What if …… one does not need to have in-depth knowledge of specific MPEG technologies / standards?

3. … one is able to start with application/business development as soon as some (reference) software becomes available?

4. … one is able to exchange applications’ underlying (reference) software with optimized one at no cost?2009/11/23-24Christian Timmerer, Klagenfurt University, Austria2

5. 2009/11/23-24Christian Timmerer, Klagenfurt University, Austria3

6. OutlineVisionOverviewArchitectureApplication Programming Interface (API)Example InstantiationsFully Interoperable StreamingIncluding MPEG-4 3D Graphics in 3rd-Party AppsSharing Protected Contents2009/11/23-24Christian Timmerer, Klagenfurt University, Austria4

7. The MXM VisionMPEG-21 Multimedia Framework

8. From Framework to Platform respecting

9. Creator and rights holders rights to exploit their works

10. End user wish to fully enjoy the benefits of digital media

11. Various value-chain player interest to provide products and services➪ DMP has specified Interoperable DRM Platform (IDP)2009/11/23-24Christian Timmerer, Klagenfurt University, Austria5“every human is potentially an element of a network involving billions of content providers, value adders, packagers, service providers, resellers, consumers ...” FrameworkPlatform. . .

12. The MXM Vision (cont’d)From Platform to Middleware enabling

13. APIs + protocols in a platform-independent way + … (see the following slides)➪ MPEG Extensible Middleware (MXM)What’s next?From Middleware to Terminal enabling

14. ITU-T: definition of IPTV infrastructure and components

15. MPEG: development of enabling technologies for, e.g., IPTV➪ Advanced IPTV Terminal (AIT)2009/11/23-24Christian Timmerer, Klagenfurt University, Austria6. . .MiddlewareTerminal

16. OverviewA set of APIs to access possibly all MPEG technologies

17. Simple methods to call complex functionalities inside MXM engines

18. “Thin” applications because the complexity is in the MXM engines

19. Replacement of MXM engines with better performing ones at no cost

20. Creation of a global market of MXM Engines, MXM Applications and MXM Devices2009/11/23-24Christian Timmerer, Klagenfurt University, Austria7

21. Overview (cont’d)The MXM standard – ISO/IEC 23006 – is subdivided in four parts:Part 1 - MXM Architecture and Technologies specifies the MXM architecture and references the technologies that are part of an MXM implementation;

22. Part 2 - MXM Application Programming Interfaces (APIs): specifies the MXM APIs;

23. Part 3 - MXM Conformance and Reference Software: specifies conformance tests and the software implementation of the standard

24. Part 4 – MXM Protocols: specifies a set of protocols enabling distributed applications to exchange information related to content items and parts thereof, including rights and protection information2009/11/23-24Christian Timmerer, Klagenfurt University, Austria8

25. MXM Application 1MXM Application 2MXMEngineAPIs‏MXM DeviceMPEG21 File EngineDigital ItemEngineREL EngineIPMPEngineOtherEnginesSecurityEngineSceneEngineContentMetadataEngineMediaFrameworkEngineOSDrivers, Accelerators, Controllers, etc.HW92009/11/23-24Christian Timmerer, Klagenfurt University, Austria

26. MXM Application 3MXM Application 1MXM Application 2MXMOrchestratorAPI‏MXM DeviceMPEG21 File EngineDigital ItemEngineREL EngineIPMPEngineOrchestratorEngineOtherEnginesSecurityEngineSceneEngineContentMetadataEngineMediaFrameworkEngineOSDrivers, Accelerators, Controllers, etc.HW102009/11/23-24Christian Timmerer, Klagenfurt University, Austria

27. MXM ProtocolsLicenseProviderDeviceContentIdentific.DeviceContentCreationDeviceEnd-UserDeviceContentProviderDeviceDRM ToolProviderDevice112009/11/23-24Christian Timmerer, Klagenfurt University, Austria

28. MXM ApplicationMXM ApplicationMXMOSMXMOSComputing PlatformComputing PlatformEnabling MXM Apps to communicate122009/11/23-24Christian Timmerer, Klagenfurt University, Austria

29. An Impressive Number of Standards…MPEG-7 VisualAdvanced Audio CodingLASeRMDSEvent ReportingReconfigurable Video CodingBIFSHE AACIPMPXDigital Item ProcessingDigital Item DeclarationFile FormatAudio Lossless CodingIntellectual Property Management and ProtectionMedia Value Chain OntologyRights Expression LanguageDigital Item IdentificationXML IPMP messagesDigital Item StreamingIPMP ComponentsDigital Item AdaptationMPQF132009/11/23-24Christian Timmerer, Klagenfurt University, Austria

30. Motivation of providing APIMPEG specifications: huge amount of technology157 standards in ISO/IEC 14496 (MPEG-4) family43 standards in ISO/IEC 15938 (MPEG-7) family33 standards in ISO/IEC 21000 (MPEG-21) family…Around 11 000 pages, 1,5 m high when printedHowever, majority of people developing MPEG related applications do not need to know that is inside the boxes, but only how to use it2009/11/23-2414Christian Timmerer, Klagenfurt University, Austria

31. The MXM Approach1. Creating wrapping libraries, called engines2. Opening the engines at ESSENTIAL points only3. Documenting the IN/OUT points in another … MPEG standard: 23006-2 MXM API- only 37 pages , - around 500 methods2009/11/23-2415Christian Timmerer, Klagenfurt University, Austria

32. List of MXM EnginesContentProtocolEngine

33. ContentSearchEngine

34. DIAEngine

35. DIDEngine

36. DISEngine

37. DomainEngine

38. EREngine

39. IPMPEngine

40. IPMPToolProtocolEngine

41. LicenseProtocolEngine

42. MediaFrameworkEngine

43. MetadataEngine

44. MPEG21FileEngine

45. MVCOEngine

46. OrchestratorEngine

47. RELEngine

48. RenderingEngine

49. SecurityEngine162009/11/23-24Christian Timmerer, Klagenfurt University, Austria

50. MXM APIs of Each Engine have been Divided into…Creation e.g.

51. Encode a raw audio track

52. Create an MPEG-7 metadata description

53. Access e.g.

54. Get data from a Digital Item

55. Decode a video

56. Editing e.g.

57. Add an elementary stream to a multiplexed content

58. Engine-specific e.g.

59. RELEngine: authorise(license, query)

60. LicenseProtocolEngine: requestLicense(licenseID, serviceURL) 172009/11/23-24Christian Timmerer, Klagenfurt University, Austria

61. Fully Interoperable Streaming ofMedia Resources in Heterogeneous Environments2009/11/23-24Christian Timmerer, Klagenfurt University, Austria18

62. Including MPEG-4 3D graphics in 3rd-Party ApplicationIncluding mp3, jpeg, mp4 video in third party applications is nowadays a beginner job. The complexity of such codecs is hidden behind a very simple communication interface once the content is decoded: matrix of pixels for images and wave samples for audio. Transposing the same principle in Computer Graphics world is a challenge due to the variety of representation forms and also the complexity and heterogeneity of data to be transferred2009/11/23-24Christian Timmerer, Klagenfurt University, Austria19By using the MXM 3DGraphicsEngine and its set of APIs, the complex integration work is simplified.With only some lines of code, Ogre3D, a very well known 3D graphics rendering engine, is transformed into an MPEG-4 3D graphics player.

63. Sharing Protected Content2009/11/23-24Christian Timmerer, Klagenfurt University, Austria20Client:MXM-based C++ client as Firefox plug-in Server:MXM-based Java serverProtocols:MXM Protocols over SOAP and XMPPhttp://www.smartrm.com

64. Join the MXM Development Team!Why you should joinPlenty of interesting work still has to be doneEnough space for student projects, master thesis, PhD thesis, etc.You can choose between Java, C++ or start a new implementation in another language!It gives you visibility on a broad set of MPEG technologiesWeb site, blog, reflector, and soon more utilitiesFriendly and collaborative environment 212009/11/23-24Christian Timmerer, Klagenfurt University, Austria

65. Thank you for your attention!... questions, comments, etc. are welcome …Ass.-Prof. Dipl.-Ing. Dr. Christian TimmererKlagenfurt University, Department of Information Technology (ITEC)Universitätsstrasse 65-67, A-9020 Klagenfurt, AUSTRIAchristian.timmerer@itec.uni-klu.ac.athttp://research.timmerer.com/Tel: +43/463/2700 3621 Fax: +43/463/2700 3699© Copyright: Christian Timmerer222009/11/23-24Christian Timmerer, Klagenfurt University, Austriahttp://mxm.wg11.sc29.org/mxm@lists.uni-klu.ac.athttp://wg11.sc29.org/mxmsvn/repos