The document provides an overview of selected current activities within MPEG, including requirements and timelines. It discusses the Mobile Visual Search work item which aims to enable efficient transmission of local image features for mobile visual search applications. It also outlines the MPEG Media Transport work item which focuses on efficient delivery of media to enable content and network adaptive streaming. Additionally, it summarizes the Advanced IPTV Terminal work item and its goal of defining elementary services and protocols to enable interoperability.

1. Overview of SelectedCurrent MPEG ActivitiesChristian TimmererKlagenfurt University (UNIKLU)  Faculty of Technical Sciences (TEWI)Department of Information Technology (ITEC)  Multimedia Communication (MMC)http://research.timmerer.com  http://blog.timmerer.com  mailto:christian.timmerer@itec.uni-klu.ac.atA report from the 91st MPEG meeting, Kyoto, JapanJanuary 2010

2. OutlineBackground / IntroductionMPEG, who the f*** is MPEG?Benefits for using MPEG technologies and contributing to MPEGSelected Current MPEG ActivitiesReq: MVS, MMT, AITSys: MPEG-M/-U/-VVid: 3DVC, HVCAud: SAOCConclusionsJanuary 20102Christian Timmerer, Klagenfurt University, Austria

3. Moving Picture Experts Group (MPEG)January 2010Christian Timmerer, Klagenfurt University, Austria4-3“Digital ContentCoding”(MPEG-1,2,4)MPEG Family of Standards: Applications and TechnologiesMPEG Standards Evolution:

4. Coding  Content Description Transactions“Media ContentDescription”(MPEG-7)VideoStorageStreamingApplicationsBroadbandObject-basedManipulationTechnologies“Transactions of Digital Items”(MPEG-21)MediaLoggingCompression

5. Coding

6. CommunicationsRepurposingEnterpriseContentMgmtAutomatic indexing

7. Multimedia search engines

8. Content-based retrieval

9. Personalization & summarizationContentAdaptationE-Commerce Of Digital ContentMPEG-1,2,4: have fueled tremendous growth in digital media content

10. MPEG-7: makes digital content searchable (value in the metadata!)

11. MPEG-21: transactions of digital itemsFlexibleBusinessModelsRights management

12. Media mining and decision support© John R. Smith, Tutorial at WWW’04

13. Moving Picture Experts Group (MPEG)January 2010Christian Timmerer, Klagenfurt University, Austria4MPEG Family of Standards: The Success Story continues …“Umbrella” standards – Multimedia Application Formats(MPEG-A)PhotoPlayerBroadcastingMPEG Standards Evolution:

14. Framework Integration  Tool SetsMiddleware Immersive User Experience PlatformMetadata CodingTools – MPEG System, Video, Audio(MPEG-B/-C/-D)Music PlayerArchivalStreamingApplicationsMultimedia Middleware(MPEG-E/-M)StreamingStreaming in Heterogeneous EnvironmentsObject-based ManipulationSurround SoundSurveillanceTechnologiesContent Sharing, the smart wayImmersive User Experience(MPEG-U/-V)Video Tool LibraryMulti-DeviceRendering + InteractionCombination of Standards / Profiles / Levels for applications

15. Payload / file / delivery formatsMultimedia Value ChainTiny, Fast Application DeploymentsPlatform(Advanced IPTV Terminal)Media Value NetworksGeneric coding toolsTV-EverywherePaymentCashingArchitecture

16. Application Programming Interfaces

17. Resource / quality management

18. Fault / integrity management

19. Protocol payload formatsVirtual WorldsStreamingInteractive Personalized TVInternet-TVWidgets & Advanced User Interaction

20. Sensory Information

21. Virtual World Object CharacteristicsMobile-TVMPEG-A: value combinations of existing standards

22. MPEG-B/-C/-D: have fueled tremendous growth in digital media content

23. MPEG-E/-M: make standards accessible enabling tiny applications

24. MPEG-U/-V: increase the Quality of Experience

25. Elementary / Aggregated Services

26. APIs, Protocols, Payload FormatsWhy using MPEG and contributing to MPEG?Why using MPEG standards?Leading edge SDO for multimedia coding and systemsOpen IPR, world, access (public), meeting, interface, useOn-going supportReference software and conformance bitstreamsavailableWhy contributing to MPEG standards?Direct, tough feedback from the experts ➪ improveVisibility of your research results ➪ impactGood networking, contact to industry ➪ research project(s) (proposals)Patents / Licensing ➪ $$$January 2010Christian Timmerer, Klagenfurt University, Austria5

27. Mobile Visual Search (MVS)January 2010Christian Timmerer, Klagenfurt University, Austria6RequirementsApplications:SnapTell service by Amazon/A9

28. Point & Find service by Nokia

29. kooaba application by kooaba ltd

30. Google Goggles application by GoogleScope: Bitstream enabling efficient transmission of local image features in mobile visual search and mobile augmented reality applications. Goals:Interoperability between visual search applications and visual database servers

31. Improve performance of mobile visual search applications

32. Facilitate efficient implementation of visual search functionality on mobile phoneshttp://www.chiariglione.org/mpeg/working_documents.htm#Explorations

33. MPEG Media Transport (MMT)January 2010Christian Timmerer, Klagenfurt University, Austria7RequirementsMultimedia ApplicationsObjectivesEfficient delivery of mediaEnable content and network adaptiveEnable cross-layer optimizationSupport integrated servicesSupport DRM toolsSupport content forwarding and retransmissionMultimedia ApplicationsMPEG Media Transport (MMT)MPEG Media Transport (MMT)MPEG Media Transport (MMT)Data Link InterfaceTimeline Call for Proposal in its next meeting in April 2010Evaluation of submissions in its July 2010 meetingPhase I: Application layer protocols to be finalized by the first quarter of 2011http://www.chiariglione.org/mpeg/working_documents.htm#Explorations

34. Advanced IPTV Terminal (AIT)January 2010Christian Timmerer, Klagenfurt University, Austria8RequirementsAim: definition of elementary services with corresponding set of protocols and APIs to enable any user in an AIT value chain to access those services in an interoperable fashion. The service definition and its composition is also within the scope of AIT.An AIT value chain is a collection of Users, including Creators, End Users and Service Providers .Service: a set of content and functions to be offered to a User within the AIT value chain.Elementary Service: a service for which protocols and APIs are defined within AIT.Aggregated Service: two or more elementary services.http://www.chiariglione.org/mpeg/working_documents.htm#Explorations

35. AIT: Elementary ServicesJanuary 2010Christian Timmerer, Klagenfurt University, Austria9Requirements

36. MPEG-M: MPEG-Extensible Middleware (MXM)A set of APIs to access possibly all MPEG technologiesSimple methods to call complex functionalities inside MXM engines“Thin” applications because the complexity is in the MXM enginesReplacement of MXM engines with better performing ones at no costCreation of a global market of MXM Engines, MXM Applications and MXM DevicesJanuary 2010Christian Timmerer, Klagenfurt University, Austria10Systemshttp://mxm.wg11.sc29.org/Part 1 – MXM Architecture and Technologies

37. Part 2 – MXM Application Programming Interfaces (APIs)

38. Part 3 – MXM Reference Software

39. Part 4 – MXM Protocols: specifies a set of protocols enabling distributed applications to exchange information related to content items and parts thereof, including rights and protection informationMPEG-U: Rich Media User Interfaces (RMUI)January 2010Christian Timmerer, Klagenfurt University, Austria11SystemsObjectivesEnable one to consistently deliver a rich experience across consumer electronics and mobile devicesBroad range of app domains (home network, broadcast, Web, mobile)Update UI continuouslyIntegration w/ media types (audio, video, text, 2D and 3D graphics) Use CasesRemote UI and customized/personalized UIUI aggregation from different applicationsWidget Communication, downloadable widget,secured widgetMultimodal widgethttp://www.chiariglione.org/mpeg/working_documents.htm#MPEG-Uhttp://www.chiariglione.org/mpeg/technologies/mpeg-u/mpu-widgets/index.htm

40. MPEG-V: Media Context and Control (MCC)January 2010Christian Timmerer, Klagenfurt University, Austria12SystemsSystem ArchitecturePt. 1: ArchitecturePt. 2: Control InformationPt. 3: Sensory InformationPt. 4: Virtual World Object CharacteristicsPt. 5: Data Formats for Interaction DevicesPt. 6: Common Types and ToolsPt. 7: Conformance and Reference Softwarehttp://www.chiariglione.org/mpeg/working_documents.htm#MPEG-V

41. 3D Video Coding (3DVC)TimelineDone: Test material definedFinal depth maps + preliminary anchor coding for new sequences (until April 2010)Final Anchor coding for new sequences (until July 2010)Draft CfP (April 2010), internalPreliminary and Final CfP (July/Oct. 2010)January 2010Christian Timmerer, Klagenfurt University, Austria13VideoState of the art:ISO/IEC 23002-3 (MPEG-C Pt. 3): 2D+Depth

42. Backward compatible w/ legacy devices

43. Agnostic of coding formats

44. Limited depth range (doesn’t handle occlusions)

45. ISO/IEC 14496-10 Amd: Multiview Video Coding (MVC)

46. Direct coding of multiple views

47. Exploits inter-camera redundancy

48. Rate of MVC encoded video is proportional to the number of viewshttp://www.chiariglione.org/mpeg/working_documents.htm#Explorations

49. High-performance Video Coding (HVC)Final CfP issued jointly: MPEG + VCEGTimeline01/22-02/15: formal registration period (33 pre-registered)02/22: coded test material shall be available at the test site03/02: Start of subjective tests04/12-13: registration/submission of documents04/17: subjective test results available04/17-23: evaluation of proposals at standardization meetingTest classes(* … only PSNR evaluation)Class A*: Cropped areas of size 2560x1600 (30fps):Class B: 1920x1080p 24 fps & 1920x1080p 50-60 fpsClass C: 832x480p 30-60 fps (WVGA)Class D: 416x240p 30-60 fps (WQVGA)Class E: 1280x720p 60fpsTesting fee: 8900 CHF plus 7.6% VATTest coordinator: VittorioBaroncini (FUB)January 2010Christian Timmerer, Klagenfurt University, Austria14VideoApplications➪ Everything but nothing new!?➪ Just more performance!?Requirements➪ Compression Performance➪ Picture formats (8Kx4K)➪ Color spaces + sampling➪ Frame rates➪ Scanning modes➪ Complexity➪ Low delay➪ Random access, trick modes➪ Error resilience➪ Buffer models➪ Interface to system layers➪ Scalable Video Codinghttp://www.chiariglione.org/mpeg/working_documents.htm#Explorations

50. Spatial Audio Object Coding (SAOC)January 2010Christian Timmerer, Klagenfurt University, Austria15AudioTransmission of high-quality multi-channel audio signals@bitrates previously used for mono/stereo signalsBackward compatibility with existing receiversMPEG-4 Object-Based Audio RepresentationBitrate & computational decoding complexity grows approximately proportionally with the number of transmitted objects

51. Structural complexity of the decoder can be an obstacle to implementation in many real-world applicationsSpatial Audio Object Coding (SAOC)High compression efficiency due to using a downmix signal plus accompanying parametric side information

52. Backward compatibility with respect to existing trans- mission structureshttp://www.chiariglione.org/mpeg/working_documents.htm#MPEG-D

53. ConclusionsWhat is MPEG and what are they doing (right now)?SDO in the area of multimedia coding & systemsMVS, MMT, AIT, MXM, RMUI, MCC, 3DVC, HVC, SAOC, etc.Long time commitment but sustainabilityNext meeting: 19-23 April 2010, Dresden (not that far away)January 2010Christian Timmerer, Klagenfurt University, Austria16

54. Thank you for your attention... questions, comments, etc. are welcome …Ass.-Prof. Dipl.-Ing. Dr. Christian TimmererKlagenfurt University, Department of Information Technology (ITEC)Universitätsstrasse 65-67, A-9020 Klagenfurt, AUSTRIAchristian.timmerer@itec.uni-klu.ac.athttp://research.timmerer.com/Tel: +43/463/2700 3621 Fax: +43/463/270099 3621© Copyright: Christian Timmerer17January 2010Christian Timmerer, Klagenfurt University, Austria