Spring 2002 Worldwide FAE Meeting Presented by: [Insert name(s)] [Insert Title of Presentation] 22 – 25 April 2002 Today, many elements exist to build an infrastructure for the delivery and consumption of multimedia content. There is, however, no 'big picture' to describe how these elements, either in existence or under development, relate to each other. The aim for MPEG-21 is to describe how these various elements fit together. Where gaps exist, MPEG-21 will recommend which new standards are required. ISO/IEC JTC 1/SC 29/WG 11 (MPEG) will then develop new standards as appropriate while other relevant standards may be developed by other bodies. These specifications will be integrated into the multimedia framework through collaboration between MPEG and these bodies. The result is an open framework for multimedia delivery and consumption, with both the content creator and content consumer as focal points. This open framework provides content creators and service providers with equal opportunities in the MPEG-21 enabled open market. This will also be to the benefit of the content consumer providing them access to a large variety of content in an interoperable manner. The vision for MPEG-21 is to define a multimedia framework to enable transparent and augmented use of multimedia resources across a wide range of networks and devices used by different communities.
Introduction MPEG stand for Moving Picture Experts Group working group of ISO/IEC in charge of the development of standards for coded representation of digital audio and video and related data. More and more digital audio - visual information exists and increasing. Increasing Internet popularity How fast and easy can desirable information be made available? More audio-visual information processing systems emerged The major MPEG standards include the following MPEG-1 ,MPEG-2 , MPEG-4 , MPEG-7 , MPEG-21
MPEG-1(11/1992) standard for lossy compression of video and audio Standardizes the storage and retrieval of moving pictures and audio storage media forms the basis for Video Cd and MP3 formats provide a video resolution of 352-by-240 at 30 frames per second (fps) This produces video quality slightly below the quality of conventional VCR videos. It viable for digital audio and TV broadcasting as well as the creation of video CDs Today various products and applications use the MPEG-1 standard especially the audio format, the most popular is MP3.
MPEG-1 file Extension The following are extension of MPEG-1 .mpg , .mpeg , .mp1 , .mp2 , .mp3 , m1v, .m1a , .mpa , mpv Extended Form Jpeg,H.261
MPEG-2(11/1994) MPEG-2 is widely used as the format of digital television signals that are broadcast by terrestrial, cable, and direct broadcast satellite TV systems. It also specifies the format of movies and other programs that are distributed on DVD and similar discs. As such, TV stations, TV receivers, DVD players, and other equipment are often designed to this standard. It provide resolutions of 720x480 and 1280x720 at 60 fps. MPEG1 is used for VCD while MPEG2 is used for DVD.
MPEG-4 (V1-09-1998 v2-11-1999) MPEG-4 was standardized in October 1998 in the ISO/IEC document 14496 The standard for multimedia for the fixed and mobile web It based on MPEG-1 and MPEG-2 and Apple QuickTime technology. Wavelet-based MPEG-4 files are smaller than JPEG or QuickTime files, They are designed to transmit video and images over a narrower bandwidth and can mix video with text, graphics and 2-D and 3-D animation layers.
MPEG-4 Standardized Method Audio-visual coding at very low bit rates Representing audio-visual objects Describing audio-visual objects in a scene Multiplexing and synchronizing the information associated with the objects Interacting with the audio-visual scene that is received by the end user
MPEG-4 Aims and Features Interoperability of products from different vendors Authors to have greater re-usability and flexibility with multimedia content produced Improvement in the management of Intellectual Property Rights Transparent information for network service providers Greater interactivity for end users. Users and content authors can manipulate rich media content (both natural and synthetic). The limits of interactivity are set by the author Users can have the ability to: Change their viewing or listening point in the scene, e.g. by navigation through a scene Drag objects in the scene to a different position
Cont… Trigger a range of events by clicking on a specific object, e.g. starting or stopping a video stream Select a language (if the option is provided by the author) Scalability Advanced compression that provides higher capacity on CDs and DVDs and also greater bandwidth (therefore more space for digital channels).
Main parts of MPEG-4 System Scene description stream, object Description stream, visual stream, Audio stream, multiplexing, synchronization MPEG-4/BiFS • A method of encoding a scene description in a binary form • Based on VRML97,which uses hierarchies and nodes, also used for Web • Allow user to change their view point in a scene or to interact with media objects Profile and Levels • Profile is a subset of the MPEG-4 system, Visual or Audio tool set and is used for specific Applications • MPEG-4 Tool set include ,Visual Profile ,Natural Profile , Synthetic and Natural, Audio Profile , Graphic Profile, Scene Graph Profile. Visual Coded representation of natural and synthetic visual objects
Cont… Audio Deals with the representation of audio objects Conformance Testing Conformance conditions for bit streams and devices Reference Software Normative and non-normative tools to validate the standard Delivery Multimedia Integrated Framework(DMIF) MPEG-4 data is authored once and delivered anywhere A set of interfaces for accessing multimedia content
Encoder,Decoder,Codecs,Players of MPEG-4 ENCODERS DICas, Studio Encode, WebCast,Paketvideo etc DECODERS EM8610 , EM8611 : For HDTV Decoder RealMagic :used with adobe premiere DV Edition Codecs DivX and XviD etc PLAYERS Mpegable X4 Live, Mpegable SDK 1.4,Mpegable Player, Nexplayer for PC and PDA ,WebCine , Osmose etc.
MPEG-7(08-2001) MPEG-7 “ Multimedia content description interface”. Provide tools for describing multimedia content MPEG-1, -2, -4: make content available; MPEG-7: makes content accessible, retrievable, filterable, manageable (via device / computer). Goal: audio-visual material as searchable as text.
MPEG-7 Tools Descriptor(Ds) Describe the syntax and semantics of audio, video and multimedia contents Description Schemes(Dss) Describe High level AV features e.g. describing region, segments,objects,events Description Definition Language(DDL) Allow new Descriptor and Description schemes to be defined and existing DSs to be modified. DDL consist XML Schema structural language components XML schema Data type language components MPEG-7 Specific extension
MPEG-7 Extension used to describe Content Form Condition for accessing the material Links to registry Intellectual property rights Price Classification Parental rating Content classification(into pre-defined categories) Storage features on the content Storage format Encoding Creation –director , title, short feature moive
MPEG-7 Extension used to describe Structural information(spatial ,teporal or spatio-temporal components) Scene cuts Segmentation in region Region motion tracking Low level features of content Colors ,textures, sound timbres, Melody description Conceptual information of the reality captured by the content Objects Events Interaction among objects User interaction (with the contents) User preferences Usage history
MPEG-7 TerminologyData, AV Content Description Definition 1..* ItemFeature, Language (DDL) defines 1..*Descriptor (D), 0..* DescriptionDescriptor value, Scheme *Description Scheme (DS), 1..* 1..*Description, describes signifies Descriptor Feature Data *..1 1..*Coded Description and ToDescription Definition Language (DDL) Human or System
MPEG-7: Major Parts Systems (ISO / IEC 15938 - 1) Description Definition Language (ISO / IEC 15938 - 2) Visual (ISO / IEC 15938 - 3) Audio (ISO / IEC 15938 - 4) Multimedia Description Schemes (ISO / IEC 15938 - 5) Reference Software
Objective of MPEG-7 Standardize content-based description for various types of audiovisual information Enable fast and efficient content searching, filtering and identification Describe several aspects of the content (low-level features, structure, semantic, models, collections, creation, etc.) Address a large range of applications (⇒ user preferences) s Types of audiovisual information: x Audio, speech x Moving video, still pictures, graphics, 3D models x Information on how objects are combined in scenes
Example of application areas Storage and retrieval of audiovisual databases (image, film, radio archives) Broadcast media selection (radio, TV programs) Surveillance(traffic control, surface transportation, production chains) E-commerce and Tele-shopping (searching for clothes / patterns) Remote sensing(cartography, ecology, natural resources management) Entertainment (searching for a game, for a karaoke) Cultural services (museums, art galleries) Journalism (searching for events, persons) Personalized news service on Internet (push media filtering) Intelligent multimedia presentations Educational applications Bio-medical applications
Example of queries Text: Find AV material with the concepts described by the text Semantic: Find AV material corresponding to the specified semantic Image: Find an image with a similar characteristic (global or local) Music: Play a few notes and search for corresponding musical pieces Motion: Find video with specific object motion trajectories
Relation content / description Description may be separated from the content AV material Description AV material AV material AV material s Description may be multiplexed with the content AV Desc AV Desc AV Desc
Type of description Information about the content: recording date & conditions, title, author, copyright, coding format, classification, etc. Information present in the content: Combination of low level and high level descriptors High level description: x Low level description Efficient and powerful 3 Generic and flexible 3 Intelligent / efficient search engine Lack of flexibility Indexing Search Feature extrac Retrieval High level Low level Recognition Recognition No restriction Efficiency process process on the search
Why do we need a standard ?Having a standard will ease the task of fast and efficient identification of contentthat is of interest to the user by s allowing the same described s allowing the same search engine content to be accessed by more and filters to identify described search engines and filters content from more sources
Scope of MPEG-7 Description Description Description generation consumption Research and Scope of MPEG-7 future competition The description generation Feature extraction, Indexing process, Annotation &Authoring tools,...) consumption Search engine, Filtering tool, Retrieval process, Browsing device, ...) are non normative parts of MPEG-7 The goal is to define the minimum that enables interoperability
MPEG-21 (11-2001) Vision To define a multimedia framework to enable transparent use of multimedia resources across a wide range of networks and devices used by different communities Purpose Enable electronic creation, delivery, trade of digital multimedia content Goals Provide access to information and services from almost anywhere at anytime with ubiquitous terminals and networks Identify, describe, manage, and protect multimedia content to support delivery chain of content creation, production, delivery, and consumption