The MPEG-21 Multimedia Framework

Slideshow Transcript

1. Slide 1: Workshop on MPEG Technologies January 2007 Marrakech, Morocco Dr. Christian Timmerer, Klagenfurt University, Austria Acknowledgments: I. Burnett, H. Hellwagner, F. Pereira, A. Vetro, R. Van de Walle
2. Slide 2: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria  Introduction and Overview  Digital Item Declaration and Identification  Digital Rights Management  Digital Item Adaptation  Digital Item Processing  Digital Item Streaming  Conclusions
3. Slide 3: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria  … to enable transparent and augmented use of multimedia resources across a wide range of networks, devices, user preferences, and communities, notably for trading (of bits)  Assumption: every human is potentially a node of a network involving billions of …  content providers  value adders  packagers  service providers Digital Item  consumers  resellers
4. Slide 4: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria MPEG-21’s goal is to create an interoperable and integrated multimedia framework in three steps: Develop “big picture”: understand how the components of the framework are related and identify where gaps in the framework exist Fill the gaps: develop new standard specifications where needed Integrate: achieve the integration of standards to support harmonized technologies for the management of multimedia content
5. Slide 5: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria What ? – Digital Items (DIs)  A Digital Item (DI) is a structured digital object with a standard representation, identification, and metadata within the MPEG-21 framework  Digital Items are “the content” Who ? – Users  A User is any entity that interacts in the MPEG-21 environment or makes use of a Digital Item  Users will assume rights and responsibilities according to their interaction with other Users  All parties that have a requirement within MPEG-21 to interact are categorized equally as Users
6. Slide 6: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria
7. Slide 7: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria Game Program – JPEG image security allows (watermarked) 15 plays before purchase Sound Files at another http://www.starwars.com site (different rates Web site STAR WARS available) link – The Motion Picture Link to the pay movie - 10 different rates Movie Digital Item dependent on network Scripts and terminal The DI is the fundamental unit for distribution and transaction within the MPEG-21 framework.
8. Slide 8: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria Digital Adaptation Processing Systems Misc Rights Pt. 7: Digital Pt. 10: Digital Pt. 9: File Pt. 8: Reference Management Item Adaptation Item Processing Format Software Amd.1: Convers. Amd.1: Add‘l Pt. 16: Binary Pt. 11: Persistent Pt. 4: IPMP Association Components And Permissions C++ bindings Format Amd.2: Dynamic Pt. 18: Digital Pt. 12: Test Bed Pt. 5: Rights Expression Lang and Distributed Item Streaming Pt. 14: Conform. Adaptation Pt. 6: Rights Pt. 15: Event Data Dictionary Reporting Amd.1: DII Pt. 17: Fragment relationship types Idenfication Vision, Declaration, and Identification Pt. 1: Vision, Technologies Pt. 2: Digital Item Pt. 3: Digital Item and Strategy Declaration Identification
9. Slide 9: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria Why declare Digital Items? Currently, multimedia applications are based on transfer / processing / presentation / ... of:  Different media types, with different representations  Still images (JPEG, JPEG2000, GIF, PNG, ...)  Video (MPEG-4, QuickTime, ...) and audio (WAV, MP3, ...)  Text (txt, doc, pdf, ...)  ...  Metadata  Descriptive information about actual data (MPEG-7, …)  DRM information (rights expressions, IPMP, …)  Configuration information (usage environment descriptions, …)  ...  But how do these elements relate to each other ?  Structure  MPEG-21 Solution: Digital Item Declaration Language (DIDL) Digital Item Declaration (DID) – instance conforming to DIDL
10. Slide 10: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria A Digital Item is ...  structured, with a standard representation, identification, and metadata Resources (e.g., MPEG-4, other/new formats) Structure Metadata (e.g., MPEG-7, other/new formats)  the fundamental unit of distribution and transaction in the multimedia framework  expressed by the Digital Item Declaration Language (DIDL), based on XML schema
11. Slide 11: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria Digital Item Declaration is described in three normative sections:  Model  Describes set of abstract terms and concepts  Digital Item is the digital representation of \"a work”  DI is the thing that is acted upon within the model  DIs are managed/handled/processed, described, exchanged, collected, ...  Representation  DID elements are represented in XML  Normative description of their syntax and semantics  Schema  Normative XML schema  Comprising entire grammar of the DID
12. Slide 12: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria <DIDL> <Item> <Descriptor> <Statement mimeType=\"text/plain\">Best of Mozart</Statement> </Descriptor> <Descriptor> <Component><Resource mimeType=\"image/jpg\" ref=\"cover.jpg\"/></Component> </Descriptor> <Item> <Descriptor> <Statement mimeType=\"text/plain\">Le nozze di Figaro KV 492, Overtüre, 4:08</Statement> </Descriptor> <Component> <Descriptor> <Statement mimeType=\"text/plain\">Bitrate 192kbps</Statement> </Descriptor> <Resource mimeType=\"audio/m4a\" ref=\"track01.m4a\"/> </Component> </Item> <!-- further items ... --> </Item> </DIDL>
13. Slide 13: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria  Scope: How to …  uniquely identify DIs and parts thereof (including resources)  uniquely identify IP related to the DIs and parts thereof (e.g., abstractions)  uniquely identify Description Schemes  use identifiers to link DIs with related information such as descriptive metadata  identify different types of DIs  Identifiers can be associated with DIs by including them in a statement element
14. Slide 14: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria  Rights Expression Language (REL)  Rights Data Dictionary (RDD)  Intellectual Property Management and Protection (IPMP) Components A flavor only – the specifications run to hundreds of pages of definitions ...
15. Slide 15: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria REL := machine-readable language that can declare rights and permissions on digital resources Goals: Provide a standard way to express rights/interests  For protection of digital contents  For privacy and use of personal data Provide a standard way to express grants of rights  Specify access and use of controls for digital content  Honor the rights, conditions, and fees specified Support guaranteed end-to-end interoperability
16. Slide 16: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria  Grant: four basic entities and their relationship Issued to Right Subject to Associated with Principal Resource Condition  Using this model, flexible rights expressions can be generated  License: grant and issuer
17. Slide 17: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria Grant: “John may play DI in 2007” <license> <grant> <keyHolder licensePartId=\"John\">…</keyHolder> Principal <mx:play/> Right <mx:diReference> Resource <mx:identifier>urn:grid:a1-abcde-1234567890-f</mx:identifier> </mx:diReference> <validityInterval> Condition <notBefore>2007-01-01T00:00:00</notBefore> <notAfter>2007-12-31T23:59:59</notAfter> </validityInterval> </grant> <issuer> Issuer <keyHolder licensePartId=“Xin”>…</keyHolder> </issuer> </license>
18. Slide 18: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria RDD := set of clear, consistent, structured, integrated, uniquely identified terms to support REL Goals: Provide a standard way to describe the semantics of terms based on their relations to other terms Support mapping/transformation of metadata from the terminology of one namespace (or authority) into that of another namespace (or authority)
19. Slide 19: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria IPMP Components := how to include IPMP information and protected parts of Digital Items in a DIDL document Goals: Encapsulates and protects a part of the hierarchy of a Digital Item Associates appropriate identification and protection information Purposely does not specify protection measures, keys, key management, trust management, encryption algorithms, certification infrastructures or other components that would also be needed as part of a complete IPMP solution
20. Slide 20: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria  Identifier  Appropriate identifier for the protected representation  E.g., dii:Identifier  Info  Information about the governance  E.g., IPMP tools, rights expressions, signature, keys, …  ContentInfo  Informtion about the governed „content“  E.g., MPEG-7 metadata  Contents IPMPDIDLChildGroup  The governed “content”  E.g., did:Item, did:Component, … Identifier Info ContentInfo Contents
21. Slide 21: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria DIA := syntax and semantics of tools that assist in the adaptation of Digital Items Goals: Satisfy transmission, storage and consumption constraints as well as Quality of Service (QoS) management Enable transparent access to (distributed) advanced multimedia content by shielding users from network and terminal installation issues Codec Format-independent mechanisms that provide support for Digital Item Adaptation in terms of:  Resource adaptation  Description adaptation  Quality of Service management The adaptation engines themselves are non-normative tools
22. Slide 22: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria User Characteristics Terminal Capabilities • User Info • Codec Capabilities • Usage Preference & History • Device Properties • Presentation Preferences • Input-Output Characteristics • Accessibility • Location fundamental input to any adaptation engine Network Characteristics Natural Environment Characteristics • Capabilities • Conditions • Location & Time • Audio-Visual
23. Slide 23: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria ... <MediaFormat> <VisualCoding> <Format href=\"urn:mpeg:mpeg7:cs:VisualCodingFormatCS:2001:2.2.2\"> <Name xml:lang=\"en\">MPEG-2 Video Main Profile @ Main Level</Name> </Format> <Frame height=\"720\" width=\"480\" rate=\"30\"/> <BitRate>5000000</BitRate> </VisualCoding> </MediaFormat> ... MPEG-7 Adaptation Engine ... <TerminalCapability xsi:type=\"CodecCapabilitiesType\"> DIA <Decoding xsi:type=\"ImageCapabilitiesType\"> <Format href=\"urn:mpeg:mpeg7:cs:VisualCodingFormatCS:2001:4\"> <mpeg7:Name xml:lang=\"en\">JPEG</mpeg7:Name> </Format> </Decoding> <Decoding xsi:type=\"VideoCapabilitiesType\"> <Format href=\"urn:mpeg:mpeg7:cs:VisualCodingFormatCS:2001:3.1.2\"> <mpeg7:Name xml:lang=\"en\"> MPEG-4 Visual Simple Profile @ Level 1 </mpeg7:Name> </Format> </Decoding> </TerminalCapability> ...
24. Slide 24: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria  XML document describing the high-level structure of a bitstream  E.g., headers, packets or layers… not bit-per-bit  BSD is not an alternative format, but an additional layer of metadata  Allows finer or coarser levels of detail, depending on the application Syntactic Description of MPEG-4 Bitstream to VOP Level Semantic Description of Segments
25. Slide 25: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria traditional Web (XML/XSLT) publishing BSD-based multimedia “publishing”
26. Slide 26: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria  AdaptationQoS  Specifies the relationship between Constraints (e.g., available bandwidth is 384kbps, terminal display is SIF), feasible adaptation operations satisfying these constraints (e.g., reduce bit-rate, spatial resolution) and associated utilities (e.g., quality at QCIF/30fps/QP=10 versus SIF/10fps/QP=15)  Universal Constraints Description (UCD)  Tool to express limitation constraints (e.g., greater than, equal to) and optimization constraints (e.g., max, min)  Metadata Adaptation  Given prior knowledge about the metadata, hint information is used to reduce the complexity of the metadata adaptation process  Session Mobility  Transfer the “state” of a Digital Item from one device to another  DIA Configuration  Guide adaptation process considering intentions of the author
27. Slide 27: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria  Amd. 1: Conversions and Permissions  Facilitates the description of conversion-related information  Enables constraints to be imposed on adaptation  Note that REL and RDD already provide tools to permit playing, modifying, and adapting - however, only with coarse control  Amd.1 of DIA essentially enables finer-grained control over the changes that can occur when playing, modifying, or adapting Digital Items and their component resources  Amd. 2: Dynamic and Distributed Adaptation  Dynamic adaptation refers to the adaptation of Digital Items according to dynamically changing usage environments  Distributed adaptation Refers to multiple adaptation steps successively performed on different MPEG-21 peers  Defines set of properties and attributes regarding fragmentation, timing, and random access point which are used for streamed processing and transport
28. Slide 28: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria DIP := allow Users to add functionality to a static DI Declaration Goals: Provide basic means for interaction with a Digital Item and its declaration Allow Users to add/select methods to be performed on Digital Items (e.g., display, select track, ...) Provide list of basic operations and means for executing User-defined operations used within methods (standard library)
29. Slide 29: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria  Digital Item Methods (DIMs) := “a list of operations”  Specified in normative language: DIM Language (DIML)  One DIML has been chosen: ECMAScript  Digital Item Base Operations (DIBOs) := set of normative basic operations on which DIMs are built  Analogous to standard library of functions of a programming language  Atomic operations  Normative, high-level interface  Implemented in any language  Access to Multimedia Middleware API  Digital Item eXtension Operations (DIXOs) := how to execute User-defined operations  Java and C++ bindings available
30. Slide 30: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria DIS := Bitstream Binding Language which describes how Digital Items can be mapped to delivery channels (e.g., MPEG-2 TS, RTP) Digital Item Delivery Channel (type O) <DIDL > <!--…--> Fragmentation Binding Goals: </DIDL> Delivery Channel (type P) Resource Fragment and insert Metadata (scheme X) (type Y) (i.e., map) into one <xx:meta > <!--…--> </xx:meta> BBL Processor of several delivery channels BBL Instance for DI <BBL> <Instance > Facilitate UMA to the serialization < !--... --> </Instance > </BBL> BBL Binding for of Digital Items BBL Binding for scheme X on stream P type Y on stream O <BBL> <Binding > BS Schema for type Y Different parts of a DI to be <BBL> < !--...--> <xs:schema > < Binding > </Binding > <!--…--> < !--...--> </BBL> </xs:schema > sent over separate delivery channels < /Binding > </BBL>
31. Slide 31: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria  Enable transparent and augmented use of multimedia resources across a wide range of networks, devices, user preferences, and communities, notably for trading (of bits).  MPEG-21 provides means for  declaring and identifying of Digital Items (DID, DII)  digital rights management (IPMP, REL, RDD)  (generic) adaptation of Digital Items according to the usage environment (DIA)  processing of Digital Items (DIP)  systems-related aspects (FF, Binary Format, DIS)  event reporting (ER)  reference software, conformance, technical reports
32. Slide 32: Dr. Christian Timmerer, Saturday, January 20, 2007 Klagenfurt University, Austria  Web Sites  Adopted MPEG standards  ISO/IEC: http://www.iso.org  MPEG standards under development, technologies, and working documents  MPEG Website: http://www.chiariglione.org/mpeg/  I. Burnett, R. Koenen, F. Pereira, R. Van de Walle (eds.), The MPEG-21 Book, Wiley, 2006  F. Pereira, J. R. Smith, A. Vetro (eds.), Special Section on MPEG-21, IEEE Transaction on Multimedia, vol. 7, no. 3, pp. 397-479, June 2005.  G. Drury, I. Burnett, MPEG-21 in a Backpack Journalism Scenario, IEEE MultiMedia, pp. 24-32, October 2005.  A. Tokmakoff, FX Nuttall, K. Ji, MPEG-21 Event Reporting: Enabling Multimedia E- Commerce, IEEE MultiMedia, pp. 50-59, October 2005.  C. Timmerer, H. Hellwagner, Interoperable Adaptive Multimedia Communication, IEEE MultiMedia, pp. 74-79, January 2005.  X. Wang, MPEG-21 Rights Expression Language: Enabling Interoperable Digital Rights Management, IEEE MultiMedia, pp. 84-87, October 2004.  A. Vetro, MPEG-21 Digital Item Adaptation: Enabling Universal Multimedia Access, IEEE MultiMedia, pp. 84-87, January 2004.  B. L. Tseng, C. Lin, J. R. Smith, Using MPEG-7 and MPEG-21 for Personalizing Video, IEEE MultiMedia, pp. 42-53, January 2004.
33. Slide 33: Contact Information: Dr. Christian Timmerer Klagenfurt University Department of Information Technology (ITEC) Universitätsstrasse 65-67 A-9020 Klagenfurt T +43 (463) 2700 3621 F +43 (469) 2700 3699 E christian.timmerer@itec.uni-klu.ac.at W http://research.timmerer.com Visit the IT Campus Carinthia http://www.it-campus.at