The MPEG-21 Multimedia Framework

3,499 views

Published on

Provides an introduction to the MPEG-21 multimedia framework aka ISO/IEC 21000.

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,499
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
0
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

The MPEG-21 Multimedia Framework

  1. 1. Dr. Christian Timmerer, Klagenfurt University, Austria Acknowledgments : I. Burnett, H. Hellwagner, F. Pereira, A. Vetro, R. Van de Walle Workshop on MPEG Technologies January 2007 Marrakech, Morocco
  2. 2. <ul><li>Introduction and Overview </li></ul><ul><li>Digital Item Declaration and Identification </li></ul><ul><li>Digital Rights Management </li></ul><ul><li>Digital Item Adaptation </li></ul><ul><li>Digital Item Processing </li></ul><ul><li>Digital Item Streaming </li></ul><ul><li>Conclusions </li></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  3. 3. <ul><li>… to enable transparent and augmented use of multimedia resources across a wide range of networks, devices, user preferences, and communities , notably for trading (of bits) </li></ul><ul><li>Assumption: every human is potentially a node of a network involving billions of … </li></ul><ul><ul><li>content providers </li></ul></ul><ul><ul><li>value adders </li></ul></ul><ul><ul><li>packagers </li></ul></ul><ul><ul><li>service providers </li></ul></ul><ul><ul><li>consumers </li></ul></ul><ul><ul><li>resellers </li></ul></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  4. 4. <ul><li>MPEG-21’s goal is to create an interoperable and integrated multimedia framework in three steps: </li></ul><ul><li>Develop “big picture” : understand how the components of the framework are related and identify where gaps in the framework exist </li></ul><ul><li>Fill the gaps : develop new standard specifications where needed </li></ul><ul><li>Integrate : achieve the integration of standards to support harmonized technologies for the management of multimedia content </li></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  5. 5. <ul><li>What ? – Digital Items (DIs) </li></ul><ul><li>A Digital Item (DI) is a structured digital object with a standard representation , identification , and metadata within the MPEG-21 framework </li></ul><ul><li>Digital Items are “ the content ” </li></ul><ul><li>Who ? – Users </li></ul><ul><li>A User is any entity that interacts in the MPEG-21 environment or makes use of a Digital Item </li></ul><ul><li>Users will assume rights and responsibilities according to their interaction with other Users </li></ul><ul><li>All parties that have a requirement within MPEG-21 to interact are categorized equally as Users </li></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  6. 6. Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  7. 7. Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria The DI is the fundamental unit for distribution and transaction within the MPEG-21 framework.
  8. 8. Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria Vision, Declaration, and Identification Digital Rights Management Adaptation Processing Systems Misc Pt. 4: IPMP Components Pt. 5: R ights E xpression L ang Pt. 6: R ights D ata D ictionary Pt. 7: D igital I tem A daptation Pt. 10: D igital I tem P rocessing Amd.1 : Convers. And Permissions Amd.2 : Dynamic and Distributed Adaptation Pt. 1: Vision, Technologies and Strategy Pt. 2: D igital I tem D eclaration Pt. 3: D igital I tem I dentification Pt. 9: File Format Pt. 16: Binary Format Pt. 18: D igital I tem S treaming Pt. 8: Reference Software Pt. 11: Persistent Association Pt. 12: Test Bed Pt. 14: Conform. Pt. 15: Event Reporting Pt. 17: Fragment Idenfication Amd.1 : Add‘l C++ bindings Amd.1 : DII relationship types
  9. 9. <ul><li>Why declare Digital Items? </li></ul><ul><li>Currently, multimedia applications are based on transfer / processing / presentation / ... of: </li></ul><ul><li>Different media types , with different representations </li></ul><ul><ul><li>Still images (JPEG, JPEG2000, GIF, PNG, ...) </li></ul></ul><ul><ul><li>Video (MPEG-4, QuickTime, ...) and audio (WAV, MP3, ...) </li></ul></ul><ul><ul><li>Text (txt, doc, pdf, ...) </li></ul></ul><ul><ul><li>... </li></ul></ul><ul><li>Metadata </li></ul><ul><ul><li>Descriptive information about actual data (MPEG-7, …) </li></ul></ul><ul><ul><li>DRM information (rights expressions, IPMP, …) </li></ul></ul><ul><ul><li>Configuration information (usage environment descriptions, …) </li></ul></ul><ul><ul><li>... </li></ul></ul><ul><li>But how do these elements relate to each other ?  Structure </li></ul><ul><li> MPEG-21 Solution: Digital Item Declaration Language (DIDL) </li></ul><ul><li>Digital Item Declaration (DID) – instance conforming to DIDL </li></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  10. 10. <ul><li>A Digital Item is ... </li></ul><ul><li>structured , with a standard representation, identification, and metadata </li></ul><ul><li>the fundamental unit of distribution and transaction in the multimedia framework </li></ul><ul><li>expressed by the Digital Item Declaration Language (DIDL) , based on XML schema </li></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria Structure Resources (e.g., MPEG-4, other/new formats) Metadata (e.g., MPEG-7, other/new formats)
  11. 11. <ul><li>Digital Item Declaration is described in three normative sections: </li></ul><ul><li>Model </li></ul><ul><ul><li>Describes set of abstract terms and concepts </li></ul></ul><ul><ul><li>Digital Item is the digital representation of &quot;a work” </li></ul></ul><ul><ul><li>DI is the thing that is acted upon within the model </li></ul></ul><ul><ul><li>DIs are managed/handled/processed, described, exchanged, collected, ... </li></ul></ul><ul><li>Representation </li></ul><ul><ul><li>DID elements are represented in XML </li></ul></ul><ul><ul><li>Normative description of their syntax and semantics </li></ul></ul><ul><li>Schema </li></ul><ul><ul><li>Normative XML schema </li></ul></ul><ul><ul><li>Comprising entire grammar of the DID </li></ul></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  12. 12. Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria < DIDL > < Item > < Descriptor > < Statement mimeType=&quot;text/plain&quot; > Best of Mozart </ Statement > </ Descriptor > < Descriptor > < Component >< Resource mimeType=&quot;image/jpg&quot; ref=&quot;cover.jpg&quot; /></ Component > </ Descriptor > < Item > < Descriptor > < Statement mimeType=&quot;text/plain&quot; > Le nozze di Figaro KV 492, Overtüre, 4:08 </ Statement > </ Descriptor > < Component > < Descriptor > < Statement mimeType=&quot;text/plain&quot; > Bitrate 192kbps </ Statement > </ Descriptor > < Resource mimeType=&quot;audio/m4a&quot; ref=&quot;track01.m4a&quot; /> </ Component > </ Item > <!-- further items ... --> </ Item > </ DIDL >
  13. 13. <ul><li>Scope: How to … </li></ul><ul><ul><li>uniquely identify DIs and parts thereof (including resources) </li></ul></ul><ul><ul><li>uniquely identify IP related to the DIs and parts thereof (e.g., abstractions) </li></ul></ul><ul><ul><li>uniquely identify Description Schemes </li></ul></ul><ul><ul><li>use identifiers to link DIs with related information such as descriptive metadata </li></ul></ul><ul><ul><li>identify different types of DIs </li></ul></ul><ul><li>Identifiers can be associated with DIs by including them in a statement element </li></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  14. 14. <ul><li>Rights Expression Language (REL) </li></ul><ul><li>Rights Data Dictionary (RDD) </li></ul><ul><li>Intellectual Property Management and Protection (IPMP) Components </li></ul><ul><li>A flavor only – the specifications run to hundreds of pages of definitions ... </li></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  15. 15. <ul><li>REL := machine-readable language that can declare rights and permissions on digital resources </li></ul><ul><li>Goals: </li></ul><ul><li>Provide a standard way to express rights/interests </li></ul><ul><ul><li>For protection of digital contents </li></ul></ul><ul><ul><li>For privacy and use of personal data </li></ul></ul><ul><li>Provide a standard way to express grants of rights </li></ul><ul><ul><li>Specify access and use of controls for digital content </li></ul></ul><ul><ul><li>Honor the rights, conditions, and fees specified </li></ul></ul><ul><li>Support guaranteed end-to-end interoperability </li></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  16. 16. <ul><li>Grant: four basic entities and their relationship </li></ul><ul><li> Using this model, flexible rights expressions can be generated </li></ul><ul><li>License: grant and issuer </li></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria Right Resource Principal Condition Associated with Subject to Issued to
  17. 17. <ul><li>Grant: “John may play DI in 2007” </li></ul><ul><li><license> </li></ul><ul><li><grant> </li></ul><ul><li><keyHolder licensePartId=&quot;John&quot;>…</keyHolder> Principal </li></ul><ul><li><mx:play/> Right </li></ul><ul><li><mx:diReference> Resource </li></ul><ul><li><mx:identifier>urn:grid:a1-abcde-1234567890-f</mx:identifier> </li></ul><ul><li></mx:diReference> </li></ul><ul><li><validityInterval> Condition </li></ul><ul><li><notBefore>2007-01-01T00:00:00</notBefore> </li></ul><ul><li><notAfter>2007-12-31T23:59:59</notAfter> </li></ul><ul><li></validityInterval> </li></ul><ul><li></grant> </li></ul><ul><li><issuer> Issuer </li></ul><ul><li><keyHolder licensePartId=“Xin”>…</keyHolder> </li></ul><ul><li></issuer> </li></ul><ul><li></license> </li></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  18. 18. <ul><li>RDD := set of clear, consistent, structured, integrated, uniquely identified terms to support REL </li></ul><ul><li>Goals: </li></ul><ul><li>Provide a standard way to describe the semantics of terms based on their relations to other terms </li></ul><ul><li>Support mapping/transformation of metadata from the terminology of one namespace (or authority) into that of another namespace (or authority) </li></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  19. 19. <ul><li>IPMP Components := how to include IPMP information and protected parts of Digital Items in a DIDL document </li></ul><ul><li>Goals: </li></ul><ul><li>Encapsulates and protects a part of the hierarchy of a Digital Item </li></ul><ul><li>Associates appropriate identification and protection information </li></ul><ul><li>Purposely does not specify protection measures , keys , key management , trust management , encryption algorithms , certification infrastructures or other components that would also be needed as part of a complete IPMP solution </li></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  20. 20. <ul><li>Identifier </li></ul><ul><ul><li>Appropriate identifier for the protected representation </li></ul></ul><ul><ul><li>E.g., dii:Identifier </li></ul></ul><ul><li>Info </li></ul><ul><ul><li>Information about the governance </li></ul></ul><ul><ul><li>E.g., IPMP tools, rights expressions, signature, keys, … </li></ul></ul><ul><li>ContentInfo </li></ul><ul><ul><li>Informtion about the governed „content“ </li></ul></ul><ul><ul><li>E.g., MPEG-7 metadata </li></ul></ul><ul><li>Contents </li></ul><ul><ul><li>The governed “content” </li></ul></ul><ul><ul><li>E.g., did:Item, did:Component, … </li></ul></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria IPMPDIDLChildGroup Identifier Info ContentInfo Contents
  21. 21. <ul><li>DIA := syntax and semantics of tools that assist in the adaptation of Digital Items </li></ul><ul><li>Goals: </li></ul><ul><li>Satisfy transmission , storage and consumption constraints as well as Quality of Service (QoS) management </li></ul><ul><li>Enable transparent access to (distributed) advanced multimedia content by shielding users from network and terminal installation issues </li></ul><ul><li>Codec Format-independent mechanisms that provide support for Digital Item Adaptation in terms of: </li></ul><ul><ul><li>Resource adaptation </li></ul></ul><ul><ul><li>Description adaptation </li></ul></ul><ul><ul><li>Quality of Service management </li></ul></ul><ul><li>The adaptation engines themselves are non-normative tools </li></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  22. 22. Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria <ul><li>Terminal Capabilities </li></ul><ul><li>Codec Capabilities </li></ul><ul><li>Device Properties </li></ul><ul><li>Input-Output Characteristics </li></ul><ul><li>User Characteristics </li></ul><ul><li>User Info </li></ul><ul><li>Usage Preference & History </li></ul><ul><li>Presentation Preferences </li></ul><ul><li>Accessibility </li></ul><ul><li>Location </li></ul><ul><li>Network Characteristics </li></ul><ul><li>Capabilities </li></ul><ul><li>Conditions </li></ul><ul><li>Natural Environment Characteristics </li></ul><ul><li>Location & Time </li></ul><ul><li>Audio-Visual </li></ul>fundamental input to any adaptation engine
  23. 23. Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria ... <TerminalCapability xsi:type=&quot;CodecCapabilitiesType&quot;> <Decoding xsi:type=&quot;ImageCapabilitiesType&quot;> <Format href=&quot;urn:mpeg:mpeg7:cs:VisualCodingFormatCS:2001:4&quot;> <mpeg7:Name xml:lang=&quot;en&quot;> JPEG </mpeg7:Name> </Format> </Decoding> <Decoding xsi:type=&quot;VideoCapabilitiesType&quot;> <Format href=&quot;urn:mpeg:mpeg7:cs:VisualCodingFormatCS:2001:3.1.2&quot;> <mpeg7:Name xml:lang=&quot;en&quot;> MPEG-4 Visual Simple Profile @ Level 1 </mpeg7:Name> </Format> </Decoding> </TerminalCapability> ... Adaptation Engine ... <MediaFormat> <VisualCoding> <Format href=&quot;urn:mpeg:mpeg7:cs:VisualCodingFormatCS:2001:2.2.2&quot;> <Name xml:lang=&quot;en&quot;> MPEG-2 Video Main Profile @ Main Level </Name> </Format> <Frame height=&quot; 720 &quot; width=&quot; 480 &quot; rate=&quot; 30 &quot;/> <BitRate> 5000000 </BitRate> </VisualCoding> </MediaFormat> ... MPEG-7 DIA
  24. 24. <ul><li>XML document describing the high-level structure of a bitstream </li></ul><ul><ul><li>E.g., headers, packets or layers… not bit-per-bit </li></ul></ul><ul><li>BSD is not an alternative format, but an additional layer of metadata </li></ul><ul><li>Allows finer or coarser levels of detail, depending on the application </li></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria Syntactic Description of MPEG-4 Bitstream to VOP Level Semantic Description of Segments
  25. 25. Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria traditional Web (XML/XSLT) publishing BSD-based multimedia “publishing”
  26. 26. <ul><li>AdaptationQoS </li></ul><ul><ul><li>Specifies the relationship between Constraints (e.g., available bandwidth is 384kbps, terminal display is SIF), feasible adaptation operations satisfying these constraints (e.g., reduce bit-rate, spatial resolution) and associated utilities (e.g., quality at QCIF/30fps/QP=10 versus SIF/10fps/QP=15) </li></ul></ul><ul><li>Universal Constraints Description (UCD) </li></ul><ul><ul><li>Tool to express limitation constraints (e.g., greater than, equal to) and optimization constraints (e.g., max, min) </li></ul></ul><ul><li>Metadata Adaptation </li></ul><ul><ul><li>Given prior knowledge about the metadata, hint information is used to reduce the complexity of the metadata adaptation process </li></ul></ul><ul><li>Session Mobility </li></ul><ul><ul><li>Transfer the “state ” of a Digital Item from one device to another </li></ul></ul><ul><li>DIA Configuration </li></ul><ul><ul><li>Guide adaptation process considering intentions of the author </li></ul></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  27. 27. <ul><li>Amd. 1: Conversions and Permissions </li></ul><ul><ul><li>Facilitates the description of conversion-related information </li></ul></ul><ul><ul><li>Enables constraints to be imposed on adaptation </li></ul></ul><ul><ul><li>Note that REL and RDD already provide tools to permit playing, modifying, and adapting - however, only with coarse control </li></ul></ul><ul><ul><li>Amd.1 of DIA essentially enables finer-grained control over the changes that can occur when playing, modifying, or adapting Digital Items and their component resources </li></ul></ul><ul><li>Amd. 2: Dynamic and Distributed Adaptation </li></ul><ul><ul><li>Dynamic adaptation refers to the adaptation of Digital Items according to dynamically changing usage environments </li></ul></ul><ul><ul><li>Distributed adaptation Refers to multiple adaptation steps successively performed on different MPEG-21 peers </li></ul></ul><ul><ul><li>Defines set of properties and attributes regarding fragmentation , timing , and random access point which are used for streamed processing and transport </li></ul></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  28. 28. <ul><li>DIP := allow Users to add functionality to a static DI Declaration </li></ul><ul><li>Goals: </li></ul><ul><li>Provide basic means for interaction with a Digital Item and its declaration </li></ul><ul><li>Allow Users to add/select methods to be performed on Digital Items (e.g., display, select track, ...) </li></ul><ul><li>Provide list of basic operations and means for executing User-defined operations used within methods (standard library) </li></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  29. 29. <ul><li>Digital Item Methods (DIMs) := “a list of operations” </li></ul><ul><ul><li>Specified in normative language: DIM Language (DIML) </li></ul></ul><ul><ul><li>One DIML has been chosen: ECMAScript </li></ul></ul><ul><li>Digital Item Base Operations (DIBOs) := set of normative basic operations on which DIMs are built </li></ul><ul><ul><li>Analogous to standard library of functions of a programming language </li></ul></ul><ul><ul><li>Atomic operations </li></ul></ul><ul><ul><li>Normative, high-level interface </li></ul></ul><ul><ul><li>Implemented in any language </li></ul></ul><ul><ul><li>Access to Multimedia Middleware API </li></ul></ul><ul><li>Digital Item eXtension Operations (DIXOs) := how to execute User-defined operations </li></ul><ul><ul><li>Java and C++ bindings available </li></ul></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  30. 30. <ul><li>DIS := Bitstream Binding Language which describes how Digital Items can be mapped to delivery channels (e.g., MPEG-2 TS, RTP) </li></ul><ul><li>Goals: </li></ul><ul><li>Fragment and insert (i.e., map) into one of several delivery channels </li></ul><ul><li>Facilitate UMA to the serialization of Digital Items </li></ul><ul><li>Different parts of a DI to be sent over separate delivery channels </li></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  31. 31. <ul><li>Enable transparent and augmented use of multimedia resources across a wide range of networks, devices, user preferences, and communities , notably for trading (of bits). </li></ul><ul><li>MPEG-21 provides means for </li></ul><ul><ul><li>declaring and identifying of Digital Items ( DID, DII ) </li></ul></ul><ul><ul><li>digital rights management ( IPMP, REL, RDD ) </li></ul></ul><ul><ul><li>(generic) adaptation of Digital Items according to the usage environment ( DIA ) </li></ul></ul><ul><ul><li>processing of Digital Items ( DIP ) </li></ul></ul><ul><ul><li>systems-related aspects (FF, Binary Format, DIS ) </li></ul></ul><ul><ul><li>event reporting (ER) </li></ul></ul><ul><ul><li>reference software, conformance, technical reports </li></ul></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  32. 32. <ul><li>Web Sites </li></ul><ul><ul><li>Adopted MPEG standards  ISO/IEC: http://www.iso.org </li></ul></ul><ul><ul><li>MPEG standards under development, technologies, and working documents  MPEG Website: http://www.chiariglione.org/mpeg/ </li></ul></ul><ul><li>I. Burnett, R. Koenen, F. Pereira, R. Van de Walle (eds.), The MPEG-21 Book , Wiley , 2006 </li></ul><ul><li>F. Pereira, J. R. Smith, A. Vetro (eds.), Special Section on MPEG-21 , IEEE Transaction on Multimedia , vol. 7, no. 3, pp. 397-479, June 2005. </li></ul><ul><li>G. Drury, I. Burnett, MPEG-21 in a Backpack Journalism Scenario , IEEE MultiMedia , pp. 24-32, October 2005. </li></ul><ul><li>A. Tokmakoff, FX Nuttall, K. Ji, MPEG-21 Event Reporting: Enabling Multimedia E-Commerce , IEEE MultiMedia , pp. 50-59, October 2005. </li></ul><ul><li>C. Timmerer, H. Hellwagner, Interoperable Adaptive Multimedia Communication , IEEE MultiMedia , pp. 74-79, January 2005. </li></ul><ul><li>X. Wang, MPEG-21 Rights Expression Language: Enabling Interoperable Digital Rights Management , IEEE MultiMedia , pp. 84-87, October 2004. </li></ul><ul><li>A. Vetro, MPEG-21 Digital Item Adaptation: Enabling Universal Multimedia Access , IEEE MultiMedia , pp. 84-87, January 2004. </li></ul><ul><li>B. L. Tseng, C. Lin, J. R. Smith, Using MPEG-7 and MPEG-21 for Personalizing Video , IEEE MultiMedia , pp. 42-53, January 2004. </li></ul>Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria
  33. 33. Contact Information : Dr. Christian Timmerer Klagenfurt University Department of Information Technology (ITEC) Universitätsstrasse 65-67 A-9020 Klagenfurt T +43 (463) 2700 3621 F +43 (469) 2700 3699 E [email_address] W http://research.timmerer.com Visit the IT Campus Carinthia http://www.it-campus.at

×