The MPEG-21 Multimedia Framework

Dr. Christian Timmerer, Klagenfurt University, Austria Acknowledgments : I. Burnett, H. Hellwagner, F. Pereira, A. Vetro, R. Van de Walle Workshop on MPEG Technologies January 2007 Marrakech, Morocco

Introduction and Overview
Digital Item Declaration and Identification
Digital Rights Management
Digital Item Adaptation
Digital Item Processing
Digital Item Streaming
Conclusions
Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria

… to enable transparent and augmented use of multimedia resources across a wide range of networks, devices, user preferences, and communities , notably for trading (of bits)
Assumption: every human is potentially a node of a network involving billions of …
content providers
value adders
packagers
service providers
consumers
resellers

MPEG-21’s goal is to create an interoperable and integrated multimedia framework in three steps:
Develop “big picture” : understand how the components of the framework are related and identify where gaps in the framework exist
Fill the gaps : develop new standard specifications where needed
Integrate : achieve the integration of standards to support harmonized technologies for the management of multimedia content

What ? – Digital Items (DIs)
A Digital Item (DI) is a structured digital object with a standard representation , identification , and metadata within the MPEG-21 framework
Digital Items are “ the content ”
Who ? – Users
A User is any entity that interacts in the MPEG-21 environment or makes use of a Digital Item
Users will assume rights and responsibilities according to their interaction with other Users
All parties that have a requirement within MPEG-21 to interact are categorized equally as Users

Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria The DI is the fundamental unit for distribution and transaction within the MPEG-21 framework.

Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria Vision, Declaration, and Identification Digital Rights Management Adaptation Processing Systems Misc Pt. 4: IPMP Components Pt. 5: R ights E xpression L ang Pt. 6: R ights D ata D ictionary Pt. 7: D igital I tem A daptation Pt. 10: D igital I tem P rocessing Amd.1 : Convers. And Permissions Amd.2 : Dynamic and Distributed Adaptation Pt. 1: Vision, Technologies and Strategy Pt. 2: D igital I tem D eclaration Pt. 3: D igital I tem I dentification Pt. 9: File Format Pt. 16: Binary Format Pt. 18: D igital I tem S treaming Pt. 8: Reference Software Pt. 11: Persistent Association Pt. 12: Test Bed Pt. 14: Conform. Pt. 15: Event Reporting Pt. 17: Fragment Idenfication Amd.1 : Add‘l C++ bindings Amd.1 : DII relationship types

Why declare Digital Items?
Currently, multimedia applications are based on transfer / processing / presentation / ... of:
Different media types , with different representations
Still images (JPEG, JPEG2000, GIF, PNG, ...)
Video (MPEG-4, QuickTime, ...) and audio (WAV, MP3, ...)
Text (txt, doc, pdf, ...)
...
Metadata
Descriptive information about actual data (MPEG-7, …)
DRM information (rights expressions, IPMP, …)
Configuration information (usage environment descriptions, …)
...
But how do these elements relate to each other ?  Structure
 MPEG-21 Solution: Digital Item Declaration Language (DIDL)
Digital Item Declaration (DID) – instance conforming to DIDL

A Digital Item is ...
structured , with a standard representation, identification, and metadata
the fundamental unit of distribution and transaction in the multimedia framework
expressed by the Digital Item Declaration Language (DIDL) , based on XML schema
Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria Structure Resources (e.g., MPEG-4, other/new formats) Metadata (e.g., MPEG-7, other/new formats)

Digital Item Declaration is described in three normative sections:
Model
Describes set of abstract terms and concepts
Digital Item is the digital representation of "a work”
DI is the thing that is acted upon within the model
DIs are managed/handled/processed, described, exchanged, collected, ...
Representation
DID elements are represented in XML
Normative description of their syntax and semantics
Schema
Normative XML schema
Comprising entire grammar of the DID

Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria < DIDL > < Item > < Descriptor > < Statement mimeType="text/plain" > Best of Mozart </ Statement > </ Descriptor > < Descriptor > < Component >< Resource mimeType="image/jpg" ref="cover.jpg" /></ Component > </ Descriptor > < Item > < Descriptor > < Statement mimeType="text/plain" > Le nozze di Figaro KV 492, Overtüre, 4:08 </ Statement > </ Descriptor > < Component > < Descriptor > < Statement mimeType="text/plain" > Bitrate 192kbps </ Statement > </ Descriptor > < Resource mimeType="audio/m4a" ref="track01.m4a" /> </ Component > </ Item >  </ Item > </ DIDL >

Scope: How to …
uniquely identify DIs and parts thereof (including resources)
uniquely identify IP related to the DIs and parts thereof (e.g., abstractions)
uniquely identify Description Schemes
use identifiers to link DIs with related information such as descriptive metadata
identify different types of DIs
Identifiers can be associated with DIs by including them in a statement element

Rights Expression Language (REL)
Rights Data Dictionary (RDD)
Intellectual Property Management and Protection (IPMP) Components
A flavor only – the specifications run to hundreds of pages of definitions ...

REL := machine-readable language that can declare rights and permissions on digital resources
Goals:
Provide a standard way to express rights/interests
For protection of digital contents
For privacy and use of personal data
Provide a standard way to express grants of rights
Specify access and use of controls for digital content
Honor the rights, conditions, and fees specified
Support guaranteed end-to-end interoperability

Grant: four basic entities and their relationship
 Using this model, flexible rights expressions can be generated
License: grant and issuer
Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria Right Resource Principal Condition Associated with Subject to Issued to

Grant: “John may play DI in 2007”
<license>
<grant>
<keyHolder licensePartId="John">…</keyHolder> Principal
<mx:play/> Right
<mx:diReference> Resource
<mx:identifier>urn:grid:a1-abcde-1234567890-f</mx:identifier>
</mx:diReference>
<validityInterval> Condition
<notBefore>2007-01-01T00:00:00</notBefore>
<notAfter>2007-12-31T23:59:59</notAfter>
</validityInterval>
</grant>
<issuer> Issuer
<keyHolder licensePartId=“Xin”>…</keyHolder>
</issuer>
</license>

RDD := set of clear, consistent, structured, integrated, uniquely identified terms to support REL
Goals:
Provide a standard way to describe the semantics of terms based on their relations to other terms
Support mapping/transformation of metadata from the terminology of one namespace (or authority) into that of another namespace (or authority)

IPMP Components := how to include IPMP information and protected parts of Digital Items in a DIDL document
Goals:
Encapsulates and protects a part of the hierarchy of a Digital Item
Associates appropriate identification and protection information
Purposely does not specify protection measures , keys , key management , trust management , encryption algorithms , certification infrastructures or other components that would also be needed as part of a complete IPMP solution

Identifier
Appropriate identifier for the protected representation
E.g., dii:Identifier
Info
Information about the governance
E.g., IPMP tools, rights expressions, signature, keys, …
ContentInfo
Informtion about the governed „content“
E.g., MPEG-7 metadata
Contents
The governed “content”
E.g., did:Item, did:Component, …
Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria IPMPDIDLChildGroup Identifier Info ContentInfo Contents

DIA := syntax and semantics of tools that assist in the adaptation of Digital Items
Goals:
Satisfy transmission , storage and consumption constraints as well as Quality of Service (QoS) management
Enable transparent access to (distributed) advanced multimedia content by shielding users from network and terminal installation issues
Codec Format-independent mechanisms that provide support for Digital Item Adaptation in terms of:
Resource adaptation
Description adaptation
Quality of Service management
The adaptation engines themselves are non-normative tools

Terminal Capabilities
Codec Capabilities
Device Properties
Input-Output Characteristics
User Characteristics
User Info
Usage Preference & History
Presentation Preferences
Accessibility
Location
Network Characteristics
Capabilities
Conditions
Natural Environment Characteristics
Location & Time
Audio-Visual
fundamental input to any adaptation engine

Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria ... <TerminalCapability xsi:type="CodecCapabilitiesType"> <Decoding xsi:type="ImageCapabilitiesType"> <Format href="urn:mpeg:mpeg7:cs:VisualCodingFormatCS:2001:4"> <mpeg7:Name xml:lang="en"> JPEG </mpeg7:Name> </Format> </Decoding> <Decoding xsi:type="VideoCapabilitiesType"> <Format href="urn:mpeg:mpeg7:cs:VisualCodingFormatCS:2001:3.1.2"> <mpeg7:Name xml:lang="en"> MPEG-4 Visual Simple Profile @ Level 1 </mpeg7:Name> </Format> </Decoding> </TerminalCapability> ... Adaptation Engine ... <MediaFormat> <VisualCoding> <Format href="urn:mpeg:mpeg7:cs:VisualCodingFormatCS:2001:2.2.2"> <Name xml:lang="en"> MPEG-2 Video Main Profile @ Main Level </Name> </Format> <Frame height=" 720 " width=" 480 " rate=" 30 "/> <BitRate> 5000000 </BitRate> </VisualCoding> </MediaFormat> ... MPEG-7 DIA

XML document describing the high-level structure of a bitstream
E.g., headers, packets or layers… not bit-per-bit
BSD is not an alternative format, but an additional layer of metadata
Allows finer or coarser levels of detail, depending on the application
Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria Syntactic Description of MPEG-4 Bitstream to VOP Level Semantic Description of Segments

Saturday, January 20, 2007 Dr. Christian Timmerer, Klagenfurt University, Austria traditional Web (XML/XSLT) publishing BSD-based multimedia “publishing”

AdaptationQoS
Specifies the relationship between Constraints (e.g., available bandwidth is 384kbps, terminal display is SIF), feasible adaptation operations satisfying these constraints (e.g., reduce bit-rate, spatial resolution) and associated utilities (e.g., quality at QCIF/30fps/QP=10 versus SIF/10fps/QP=15)
Universal Constraints Description (UCD)
Tool to express limitation constraints (e.g., greater than, equal to) and optimization constraints (e.g., max, min)
Metadata Adaptation
Given prior knowledge about the metadata, hint information is used to reduce the complexity of the metadata adaptation process
Session Mobility
Transfer the “state ” of a Digital Item from one device to another
DIA Configuration
Guide adaptation process considering intentions of the author

Amd. 1: Conversions and Permissions
Facilitates the description of conversion-related information
Enables constraints to be imposed on adaptation
Note that REL and RDD already provide tools to permit playing, modifying, and adapting - however, only with coarse control
Amd.1 of DIA essentially enables finer-grained control over the changes that can occur when playing, modifying, or adapting Digital Items and their component resources
Amd. 2: Dynamic and Distributed Adaptation
Dynamic adaptation refers to the adaptation of Digital Items according to dynamically changing usage environments
Distributed adaptation Refers to multiple adaptation steps successively performed on different MPEG-21 peers
Defines set of properties and attributes regarding fragmentation , timing , and random access point which are used for streamed processing and transport

DIP := allow Users to add functionality to a static DI Declaration
Goals:
Provide basic means for interaction with a Digital Item and its declaration
Allow Users to add/select methods to be performed on Digital Items (e.g., display, select track, ...)
Provide list of basic operations and means for executing User-defined operations used within methods (standard library)

Digital Item Methods (DIMs) := “a list of operations”
Specified in normative language: DIM Language (DIML)
One DIML has been chosen: ECMAScript
Digital Item Base Operations (DIBOs) := set of normative basic operations on which DIMs are built
Analogous to standard library of functions of a programming language
Atomic operations
Normative, high-level interface
Implemented in any language
Access to Multimedia Middleware API
Digital Item eXtension Operations (DIXOs) := how to execute User-defined operations
Java and C++ bindings available

DIS := Bitstream Binding Language which describes how Digital Items can be mapped to delivery channels (e.g., MPEG-2 TS, RTP)
Goals:
Fragment and insert (i.e., map) into one of several delivery channels
Facilitate UMA to the serialization of Digital Items
Different parts of a DI to be sent over separate delivery channels

Enable transparent and augmented use of multimedia resources across a wide range of networks, devices, user preferences, and communities , notably for trading (of bits).
MPEG-21 provides means for
declaring and identifying of Digital Items ( DID, DII )
digital rights management ( IPMP, REL, RDD )
(generic) adaptation of Digital Items according to the usage environment ( DIA )
processing of Digital Items ( DIP )
systems-related aspects (FF, Binary Format, DIS )
event reporting (ER)
reference software, conformance, technical reports

Web Sites
Adopted MPEG standards  ISO/IEC: http://www.iso.org
MPEG standards under development, technologies, and working documents  MPEG Website: http://www.chiariglione.org/mpeg/
I. Burnett, R. Koenen, F. Pereira, R. Van de Walle (eds.), The MPEG-21 Book , Wiley , 2006
F. Pereira, J. R. Smith, A. Vetro (eds.), Special Section on MPEG-21 , IEEE Transaction on Multimedia , vol. 7, no. 3, pp. 397-479, June 2005.
G. Drury, I. Burnett, MPEG-21 in a Backpack Journalism Scenario , IEEE MultiMedia , pp. 24-32, October 2005.
A. Tokmakoff, FX Nuttall, K. Ji, MPEG-21 Event Reporting: Enabling Multimedia E-Commerce , IEEE MultiMedia , pp. 50-59, October 2005.
C. Timmerer, H. Hellwagner, Interoperable Adaptive Multimedia Communication , IEEE MultiMedia , pp. 74-79, January 2005.
X. Wang, MPEG-21 Rights Expression Language: Enabling Interoperable Digital Rights Management , IEEE MultiMedia , pp. 84-87, October 2004.
A. Vetro, MPEG-21 Digital Item Adaptation: Enabling Universal Multimedia Access , IEEE MultiMedia , pp. 84-87, January 2004.
B. L. Tseng, C. Lin, J. R. Smith, Using MPEG-7 and MPEG-21 for Personalizing Video , IEEE MultiMedia , pp. 42-53, January 2004.

Contact Information : Dr. Christian Timmerer Klagenfurt University Department of Information Technology (ITEC) Universitätsstrasse 65-67 A-9020 Klagenfurt T +43 (463) 2700 3621 F +43 (469) 2700 3699 E [email_address] W http://research.timmerer.com Visit the IT Campus Carinthia http://www.it-campus.at

The MPEG-21 Multimedia Framework

by Alpen-Adria-Universität Klagenfurt on Oct 08, 2008

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 2

Statistics

The MPEG-21 Multimedia Framework — Presentation Transcript