Overview of Selected Current MPEG Activities


Published on

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Efficient delivery of mediaover packet based networks in adaptive progressive download/streaming fashion Enable content and network adaptive and flexible error protection and recoveryCross-layer optimization to improve the Quality of Service/ExperienceSupport integrated services with multiple components and/or hybrid delivery environmentSupport efficient signaling, delivery and utilization of multiple content protection and rights management tools.Support of efficient content forwarding and retransmission.
  • Overview of Selected Current MPEG Activities

    1. 1. Overview of SelectedCurrent MPEG Activities<br />Christian Timmerer<br />Klagenfurt University (UNIKLU)  Faculty of Technical Sciences (TEWI)<br />Department of Information Technology (ITEC)  Multimedia Communication (MMC)<br />http://research.timmerer.com  http://blog.timmerer.com  mailto:christian.timmerer@itec.uni-klu.ac.at<br />A report from the 91st MPEG meeting, Kyoto, Japan<br />January 2010<br />
    2. 2. Outline<br />Background / Introduction<br />MPEG, who the f*** is MPEG?<br />Benefits for using MPEG technologies and contributing to MPEG<br />Selected Current MPEG Activities<br />Req: MVS, MMT, AIT<br />Sys: MPEG-M/-U/-V<br />Vid: 3DVC, HVC<br />Aud: SAOC<br />Conclusions<br />January 2010<br />2<br />Christian Timmerer, Klagenfurt University, Austria<br />
    3. 3. Moving Picture Experts Group (MPEG)<br />January 2010<br />Christian Timmerer, Klagenfurt University, Austria<br />4-3<br />“Digital Content<br />Coding”<br />(MPEG-1,2,4)<br />MPEG Family of Standards: Applications and Technologies<br /><ul><li>MPEG Standards Evolution:
    4. 4. Coding  Content Description Transactions</li></ul>“Media Content<br />Description”<br />(MPEG-7)<br />Video<br />Storage<br />Streaming<br /><ul><li>Applications</li></ul>Broadband<br />Object-based<br />Manipulation<br /><ul><li>Technologies</li></ul>“Transactions of<br /> Digital Items”<br />(MPEG-21)<br />Media<br />Logging<br /><ul><li>Compression
    5. 5. Coding
    6. 6. Communications</li></ul>Repurposing<br />Enterprise<br />Content<br />Mgmt<br /><ul><li>Automatic indexing
    7. 7. Multimedia search engines
    8. 8. Content-based retrieval
    9. 9. Personalization & summarization</li></ul>Content<br />Adaptation<br />E-Commerce <br />Of Digital <br />Content<br /><ul><li>MPEG-1,2,4: have fueled tremendous growth in digital media content
    10. 10. MPEG-7: makes digital content searchable (value in the metadata!)
    11. 11. MPEG-21: transactions of digital items</li></ul>Flexible<br />Business<br />Models<br /><ul><li>Rights management
    12. 12. Media mining and decision support</li></ul>© John R. Smith, Tutorial at WWW’04<br />
    13. 13. Moving Picture Experts Group (MPEG)<br />January 2010<br />Christian Timmerer, Klagenfurt University, Austria<br />4<br />MPEG Family of Standards: The Success Story continues …<br />“Umbrella” standards – Multimedia Application Formats<br />(MPEG-A)<br />PhotoPlayer<br />Broadcasting<br /><ul><li>MPEG Standards Evolution:
    14. 14. Framework Integration  Tool SetsMiddleware Immersive User Experience Platform</li></ul>Metadata Coding<br />Tools – MPEG System, Video, Audio<br />(MPEG-B/-C/-D)<br />Music Player<br />Archival<br />Streaming<br /><ul><li>Applications</li></ul>Multimedia Middleware<br />(MPEG-E/-M)<br />Streaming<br />Streaming in Heterogeneous Environments<br />Object-based Manipulation<br />Surround Sound<br />Surveillance<br /><ul><li>Technologies</li></ul>Content Sharing, the smart way<br />Immersive User Experience<br />(MPEG-U/-V)<br />Video Tool Library<br />Multi-DeviceRendering + Interaction<br /><ul><li>Combination of Standards / Profiles / Levels for applications
    15. 15. Payload / file / delivery formats</li></ul>Multimedia Value Chain<br />Tiny, Fast Application Deployments<br />Platform<br />(Advanced IPTV Terminal)<br />Media Value Networks<br /><ul><li>Generic coding tools</li></ul>TV-Everywhere<br />PaymentCashing<br /><ul><li>Architecture
    16. 16. Application Programming Interfaces
    17. 17. Resource / quality management
    18. 18. Fault / integrity management
    19. 19. Protocol payload formats</li></ul>Virtual Worlds<br />Streaming<br />Interactive Personalized TV<br />Internet-TV<br /><ul><li>Widgets & Advanced User Interaction
    20. 20. Sensory Information
    21. 21. Virtual World Object Characteristics</li></ul>Mobile-TV<br /><ul><li>MPEG-A: value combinations of existing standards
    22. 22. MPEG-B/-C/-D: have fueled tremendous growth in digital media content
    23. 23. MPEG-E/-M: make standards accessible enabling tiny applications
    24. 24. MPEG-U/-V: increase the Quality of Experience
    25. 25. Elementary / Aggregated Services
    26. 26. APIs, Protocols, Payload Formats</li></li></ul><li>Why using MPEG and contributing to MPEG?<br />Why using MPEG standards?<br />Leading edge SDO for multimedia coding and systems<br />Open IPR, world, access (public), meeting, interface, use<br />On-going support<br />Reference software and conformance bitstreamsavailable<br />Why contributing to MPEG standards?<br />Direct, tough feedback from the experts ➪ improve<br />Visibility of your research results ➪ impact<br />Good networking, contact to industry ➪ research project(s) (proposals)<br />Patents / Licensing ➪ $$$<br />January 2010<br />Christian Timmerer, Klagenfurt University, Austria<br />5<br />
    27. 27. Mobile Visual Search (MVS)<br />January 2010<br />Christian Timmerer, Klagenfurt University, Austria<br />6<br />Requirements<br />Applications:<br /><ul><li>SnapTell service by Amazon/A9
    28. 28. Point & Find service by Nokia
    29. 29. kooaba application by kooaba ltd
    30. 30. Google Goggles application by Google</li></ul>Scope: Bitstream enabling efficient transmission of local image features in mobile visual search and mobile augmented reality applications. <br />Goals:<br /><ul><li>Interoperability between visual search applications and visual database servers
    31. 31. Improve performance of mobile visual search applications
    32. 32. Facilitate efficient implementation of visual search functionality on mobile phones</li></ul>http://www.chiariglione.org/mpeg/working_documents.htm#Explorations<br />
    33. 33. MPEG Media Transport (MMT)<br />January 2010<br />Christian Timmerer, Klagenfurt University, Austria<br />7<br />Requirements<br />Multimedia Applications<br />Objectives<br />Efficient delivery of media<br />Enable content and network adaptive<br />Enable cross-layer optimization<br />Support integrated services<br />Support DRM tools<br />Support content forwarding and retransmission<br />Multimedia Applications<br />MPEG Media Transport (MMT)<br />MPEG Media Transport (MMT)<br />MPEG Media Transport (MMT)<br />Data Link Interface<br />Timeline <br />Call for Proposal in its next meeting in April 2010<br />Evaluation of submissions in its July 2010 meeting<br />Phase I: Application layer protocols to be finalized by the first quarter of 2011<br />http://www.chiariglione.org/mpeg/working_documents.htm#Explorations<br />
    34. 34. Advanced IPTV Terminal (AIT)<br />January 2010<br />Christian Timmerer, Klagenfurt University, Austria<br />8<br />Requirements<br />Aim: definition of elementary services with corresponding set of protocols and APIs to enable any user in an AIT value chain to access those services in an interoperable fashion. The service definition and its composition is also within the scope of AIT.<br />An AIT value chain is a collection of Users, including Creators, End Users and Service Providers .<br />Service: a set of content and functions to be offered to a User within the AIT value chain.<br />Elementary Service: a service for which protocols and APIs are defined within AIT.<br />Aggregated Service: two or more elementary services.<br />http://www.chiariglione.org/mpeg/working_documents.htm#Explorations<br />
    35. 35. AIT: Elementary Services<br />January 2010<br />Christian Timmerer, Klagenfurt University, Austria<br />9<br />Requirements<br />
    36. 36. MPEG-M: MPEG-Extensible Middleware (MXM)<br />A set of APIs to access possibly all MPEG technologies<br />Simple methods to call complex functionalities inside MXM engines<br />“Thin” applications because the complexity is in the MXM engines<br />Replacement of MXM engines with better performing ones at no cost<br />Creation of a global market of MXM Engines, MXM Applications and MXM Devices<br />January 2010<br />Christian Timmerer, Klagenfurt University, Austria<br />10<br />Systems<br />http://mxm.wg11.sc29.org/<br /><ul><li>Part 1 – MXM Architecture and Technologies
    37. 37. Part 2 – MXM Application Programming Interfaces (APIs)
    38. 38. Part 3 – MXM Reference Software
    39. 39. Part 4 – MXM Protocols: specifies a set of protocols enabling distributed applications to exchange information related to content items and parts thereof, including rights and protection information</li></li></ul><li>MPEG-U: Rich Media User Interfaces (RMUI)<br />January 2010<br />Christian Timmerer, Klagenfurt University, Austria<br />11<br />Systems<br />Objectives<br />Enable one to consistently deliver a rich experience across consumer electronics and mobile devices<br />Broad range of app domains (home network, broadcast, Web, mobile)<br />Update UI continuously<br />Integration w/ media types (audio, video, text, 2D and 3D graphics) <br />Use Cases<br />Remote UI and customized/personalized UI<br />UI aggregation from different applications<br />Widget Communication, downloadable widget,secured widget<br />Multimodal widget<br />http://www.chiariglione.org/mpeg/working_documents.htm#MPEG-U<br />http://www.chiariglione.org/mpeg/technologies/mpeg-u/mpu-widgets/index.htm<br />
    40. 40. MPEG-V: Media Context and Control (MCC)<br />January 2010<br />Christian Timmerer, Klagenfurt University, Austria<br />12<br />Systems<br />System Architecture<br />Pt. 1: Architecture<br />Pt. 2: Control Information<br />Pt. 3: Sensory Information<br />Pt. 4: Virtual World Object Characteristics<br />Pt. 5: Data Formats for Interaction Devices<br />Pt. 6: Common Types and Tools<br />Pt. 7: Conformance and Reference Software<br />http://www.chiariglione.org/mpeg/working_documents.htm#MPEG-V<br />
    41. 41. 3D Video Coding (3DVC)<br />Timeline<br />Done: Test material defined<br />Final depth maps + preliminary anchor coding for new sequences (until April 2010)<br />Final Anchor coding for new sequences (until July 2010)<br />Draft CfP (April 2010), internal<br />Preliminary and Final CfP (July/Oct. 2010)<br />January 2010<br />Christian Timmerer, Klagenfurt University, Austria<br />13<br />Video<br />State of the art:<br /><ul><li>ISO/IEC 23002-3 (MPEG-C Pt. 3): 2D+Depth
    42. 42. Backward compatible w/ legacy devices
    43. 43. Agnostic of coding formats
    44. 44. Limited depth range (doesn’t handle occlusions)
    45. 45. ISO/IEC 14496-10 Amd: Multiview Video Coding (MVC)
    46. 46. Direct coding of multiple views
    47. 47. Exploits inter-camera redundancy
    48. 48. Rate of MVC encoded video is proportional to the number of views</li></ul>http://www.chiariglione.org/mpeg/working_documents.htm#Explorations<br />
    49. 49. High-performance Video Coding (HVC)<br />Final CfP issued jointly: MPEG + VCEG<br />Timeline<br />01/22-02/15: formal registration period (33 pre-registered)<br />02/22: coded test material shall be available at the test site<br />03/02: Start of subjective tests<br />04/12-13: registration/submission of documents<br />04/17: subjective test results available<br />04/17-23: evaluation of proposals at standardization meeting<br />Test classes(* … only PSNR evaluation)<br />Class A*: Cropped areas of size 2560x1600 (30fps):<br />Class B: 1920x1080p 24 fps & 1920x1080p 50-60 fps<br />Class C: 832x480p 30-60 fps (WVGA)<br />Class D: 416x240p 30-60 fps (WQVGA)<br />Class E: 1280x720p 60fps<br />Testing fee: 8900 CHF plus 7.6% VAT<br />Test coordinator: VittorioBaroncini (FUB)<br />January 2010<br />Christian Timmerer, Klagenfurt University, Austria<br />14<br />Video<br />Applications<br />➪ Everything but nothing new!?<br />➪ Just more performance!?<br />Requirements<br />➪ Compression Performance<br />➪ Picture formats (8Kx4K)<br />➪ Color spaces + sampling<br />➪ Frame rates<br />➪ Scanning modes<br />➪ Complexity<br />➪ Low delay<br />➪ Random access, trick modes<br />➪ Error resilience<br />➪ Buffer models<br />➪ Interface to system layers<br />➪ Scalable Video Coding<br />http://www.chiariglione.org/mpeg/working_documents.htm#Explorations<br />
    50. 50. Spatial Audio Object Coding (SAOC)<br />January 2010<br />Christian Timmerer, Klagenfurt University, Austria<br />15<br />Audio<br />Transmission of high-quality multi-channel audio signals<br />@bitrates previously used for mono/stereo signals<br />Backward compatibility with existing receivers<br />MPEG-4 Object-Based Audio Representation<br /><ul><li>Bitrate & computational decoding complexity grows approximately proportionally with the number of transmitted objects
    51. 51. Structural complexity of the decoder can be an obstacle to implementation in many real-world applications</li></ul>Spatial Audio Object Coding (SAOC)<br /><ul><li>High compression efficiency due to using a downmix signal plus accompanying parametric side information
    52. 52. Backward compatibility with respect to existing trans- mission structures</li></ul>http://www.chiariglione.org/mpeg/working_documents.htm#MPEG-D<br />
    53. 53. Conclusions<br />What is MPEG and what are they doing (right now)?<br />SDO in the area of multimedia coding & systems<br />MVS, MMT, AIT, MXM, RMUI, MCC, 3DVC, HVC, SAOC, etc.<br />Long time commitment but sustainability<br />Next meeting: 19-23 April 2010, Dresden (not that far away)<br />January 2010<br />Christian Timmerer, Klagenfurt University, Austria<br />16<br />
    54. 54. Thank you for your attention<br />... questions, comments, etc. are welcome …<br />Ass.-Prof. Dipl.-Ing. Dr. Christian Timmerer<br />Klagenfurt University, Department of Information Technology (ITEC)<br />Universitätsstrasse 65-67, A-9020 Klagenfurt, AUSTRIA<br />christian.timmerer@itec.uni-klu.ac.at<br />http://research.timmerer.com/<br />Tel: +43/463/2700 3621 Fax: +43/463/270099 3621<br />© Copyright: Christian Timmerer<br />17<br />January 2010<br />Christian Timmerer, Klagenfurt University, Austria<br />