MPEG-4 Developments

3,092 views

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,092
On SlideShare
0
From Embeds
0
Number of Embeds
2,751
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

MPEG-4 Developments

  1. 1. Overview of MPEG-4 Developments Olivier Avaro Chairman MPEG Systems Deutsche Telekom - Berkom Davic European Information Day Geneva, 12 January 1999 Berkom ===!" §
  2. 2. Goals of the presentation • Give a report of the status of MPEG-4 developments. • Give a report of the status of the AIC Initiative. • Some special highlights : – Video and Systems tools, – Carriage of MPEG-4 content over MPEG-2 and IP. Berkom ===!" §
  3. 3. Messages • MPEG-4 is not meant to replace existing architectures (ex : MPEG-2 or IP based architecture). • MPEG-4 augments existing architecture’s strong points with new concepts (ex : Interactive Scene Description, new content types, Java APIs). • MPEG-4 extends existing architecture’s strong points to new environments (ex : delivery independent representation of the content). Berkom ===!" §
  4. 4. Overview • Introduction to MPEG-4 : – History, Requirements and Architecture, – MPEG-4 Video Version 1 tools, – MPEG-4 Systems Version 1 tools. • Carriage of MPEG-4 over MPEG-2 and IP. • Introduction to the AIC Initiative : – Objectives and architecture, – Applications, – Work plan. • Annexes – MPEG-4 Object Content Information (OCI) – MPEG-4 Intellectual Property Management and Protection (IPMP) Berkom ===!" §
  5. 5. A bit of history on MPEG-4 • Participants MPEG: Consumer Electronics, Computer Industry, Telecom Operators, Academia. • 1993 : started as ‘Very Low Bitrate Audio-visual Coding’. • 1994: goal changed to ‘Coding of Audio-visual Objects ’. • Now: Version 1 is ready; released for IS ballot at the end of the month. Version 2 released for CD ballot in March. ‘TV/film’ AV- data From the MPEG-4 Call for Proposals (drafted 1994) interactivity Wireless ‘Computer’ ‘Telecom’ Berkom ===!" §
  6. 6. Requirements • Provide common technology for many types of services : interactive, broadcast, conversational. • Increase client-side and client-server interactivity. • Integrate natural and synthetic content. • Cover a wide range of access conditions. • Provide serious solutions for the management and protection of intellectual property. Berkom ===!" §
  7. 7. MPEG-4 Principles • Audio-visual scenes made of audio-visual objects composed together according to a scene description : – allows interaction with elements within the audio-visual scene, – coding scheme can differ for individual objects, – allows easy re-use of audio-visual content. • Audio-visual objects can be of different nature : – audio (single or multi-channel) or video (arbitrary shape or rectangular), – natural (natural audio or video) or synthetic (text & graphics, animated faces, synthetic music), – 2D (Web like pages) or 3D (spatialized sound, 3D virtual world), – streamed (video movie) or downloaded (audio jingle). Berkom ===!" §
  8. 8. MPEG-4 Principles • Scene Description provides : – the spatial/temporal relationship between the audiovisual objects (2D, 3D, mixed 2D and 3D scene description), – the behavior and interactivity of the audio-visual objects and scenes, – protocols to modify and animate the scene in time, – a binary encoding for the scene. • These principles are independent of the bit rate. Berkom ===!" §
  9. 9. Architecture Decoding TransMux FlexMux Composition and N Rendering Primitive e t AV Objects ... w ... ... o r Ex: MPEG-2 Elementary Transport Streams Scene Description Audiovisual Interactive Display and User k Information Scene Interaction Object Descriptor Berkom DAI ===!" §
  10. 10. Example of an MPEG-4 Audiovisual Scene (1) 2D Audio-visual scene 2D Audio-visual scene Audio and Video + Scrolling Text Audio and Video + Still Images and Still Images Berkom ===!" §
  11. 11. Example of an MPEG-4 Audiovisual Scene (2) 3D Audio-visual scene 3D World + arbitrary shaped video + still images + 3D Objects Berkom ===!" §
  12. 12. MPEG-4 Visual Version 1 Tools • Natural video from 10 kbit/s - 10 Mbit/s – Rectangular or arbitrary shape objects, – Scalable, – Interlaced and Progressive scan supported, – ‘Sprites’ (e.g. backgrounds): send once, warp after. • Computer-generated visual information – facial animation, – animated meshes with moving texture, – animated text and graphics. • All these objects can be synchronised together. Berkom ===!" §
  13. 13. Example of MPEG-4 Video Objects Arbitrary shape Animated Face Rectangular shape video object video object Berkom ===!" §
  14. 14. Visual Version 2 Tools • Body animation : to augment the face animation tool. • Compression of 3D meshes with texture : – games, higher quality virtual environments. • Higher Quality Video – currently under discussion. • Ready end ‘99 • Note that V.2 is a compatible extension of V.1 Berkom ===!" §
  15. 15. MPEG-4 Systems Version 1Tools • Decoder Model – Predictable behavior of a decoder and decoded content, – Management of decoder resources (buffer and memory), – Tight synchronization of Audio, Video and synthetic elements. • Object Descriptor Protocol – Identification (location and stream type) of Elementary Streams (content in the same scene can come from different sources), – Association of Elementary Streams (alternative content, scalable content, synchronization hierarchy), – Association of the scene description and the streams, – Object Content Information, – Intellectual property protection and management. Berkom ===!" §
  16. 16. MPEG-4 Systems Version 1Tools • BIFS Scene Description – VRML concepts : set of nodes to represent the primitive scene objects to be composed, the scene graph constructs, the behavior and interactivity + Integration of streams, + 2D capabilities, + Integration of 2D and 3D, + Advanced Audio Features, + Timing model, + BIFS - Update protocol to modify the scene in time, + BIFS - Anim protocol to animate the scene in time, + Compression efficiency (BIFS is Binary Format for Scene). Berkom ===!" §
  17. 17. Example of MPEG-4 Scenes (1) 2D Audio-visual scene Audio and Video + Still Images 2D Audio-visual scene Animated Text+ Video + Still Images Berkom ===!" §
  18. 18. Example of MPEG-4 Scenes (2) 2D and 3D Graphics Primitive Scene with Face Object Web-like MPEG-4 Scene Multi audio and video MPEG-4 Scene with text Berkom ===!" §
  19. 19. MPEG-4 Systems Version 2 Tools • File format – allows interchange and streaming of MPEG-4 content. • Advanced BIFS – New BIFS functionality : environmental spatialization, integration of HTML and BIFS, chroma keying, application signaling, – Advanced BIFS coding : PROTOs and ExternProtos, Integration of Mesh, coding of MField. • MPEG-J – The MPEG-4 ‘Application Execution Engine’ – MPEG-4 specific API for interaction with the scene (complicated content behavior), with the network / terminal / UI resources. Berkom ===!" §
  20. 20. MPEG-J Architecture MPEG-J Run Time Environment Java MPEG-let Terminal Java Virtual Machine DMIF Application Decoders Compositor Interface Berkom ===!" §
  21. 21. Profiles and Levels • Most applications only need a part of the MPEG-4 tool set. • Profiles define subsets useful for a large class of applications/services. • Types of Profiles: – Scene description (e.g. behaviour), – Object Descriptor (mainly timing models), – Audio (natural and synthetic): types of objects, – Visual (natural and synthetic): types of objects, – Graphics. • Levels limit the number of objects and complexity • Profiles/Levels can be easily added when need arises Berkom ===!" §
  22. 22. Profiles and Levels MPEG-4 device Scene Description Profiles Main speech simple 2D 3D Audio Audio Profiles core Core hybrid 3D FA 2D Object Descriptor Profile Visual Graphics Profiles Profiles Media Profiles Berkom ===!" §
  23. 23. Carriage of MPEG-4 Content • Transport of MPEG-4 content : left to other bodies – Choose IP, MPEG-2 TS, ATM AAL5, … • DMIF abstracts Application from delivery of data and provide a uniform walkthrough to access content – e.g. Broadcast, Internet protocol stack • Create content once, play on any network configuration Berkom ===!" §
  24. 24. Delivery Framework SL-Packetized Streams Media DAI FlexMux FlexMux FlexMux Streams Protection sL Protection sL Delivery (RTP) (PES) AAL2 H223 …. UDP MPEG2 ATM PSTN Mux sub Layer IP TS TransMux Streams Berkom ===!" §
  25. 25. MPEG-4 over MPEG-2 and IP • Technical issues to be addressed : – Transport/Multiplexing/Packetization (ex : FlexMux, PES only or IETF RTP multiplex), – Synchronization (ex : Sync Layer, RTP sync, MPEG-2 sync), – Identification and association of streams (ex : Object Descriptor, SDP). • Solved by making tradeoff between : – How far the integration should be done (ex : using MPEG-2 content or RTP streams in MPEG-4 scenes, using the MPEG-4 content types in MPEG-2 or RTP applications), – How much the duplication of functionality can be reduced. Berkom ===!" §
  26. 26. MPEG-4 over MPEG-2 and IP • Examples – Blind transmission of MPEG-4 Programs • MPEG-4 content separate from others MPEG-2/RTP streams – MPEG-4 Content (ES) in an MPEG-2 Systems /RTP application • adding MPEG-4 content types to MPEG-2/RTP – MPEG-4 ES + Systems in MPEG-2/RTP environment • Adding MPEG-4 Interactive programming to MPEG-2/RTP • MPEG-2/RTP ‘Objects’ may be used in the MPEG-4 application • On-going activity within MPEG and IETF that should lead for end 99 to : – addenda to MPEG-2 and MPEG-4, – IETF RFC. Berkom ===!" §
  27. 27. AIC-I Charter • Goal: Harmonize standards efforts and enable interactive 2D, 3D and streaming content for commerce and entertainment on set top devices. • The Advanced Interactive Content Initiative (AIC Initiative) applied to be an industry consortium operating under the Industry Technical Agreement (ITA) of the International Electro technical Commission (IEC). • The purpose of the AIC Initiative is the development of interactive content specifications for user devices to offer a range of basic through advanced interactive applications using 3D as well as 2D content, in stored and streamed form. Berkom ===!" §
  28. 28. AIC-I Applications • Advanced Electronic Program Guide. • Buy-me-button. • Data Ticker. • Program Enhancement. • Interactive Home Shopping. • Interactive Commercial. • Interactive Entertainment. • Demographic Programming. Berkom ===!" §
  29. 29. AIC-I Principles • Delivery in various transport environments – including MPEG-2 and IP environments, – and combinations of broadcast/interactive delivery. • Presentation engine : – One specification for integration of MPEG-4, ‘X-HTML’, VRML content. • Application execution engine – Not the priority right now. Berkom ===!" §
  30. 30. AIC-I Workplan • 20th December: Delivery of Version 1.0. • 8th-9th February: Delivery of Version 1.1. • 21st-22nd of March: Field trial #1. Delivery of Version 1.2. • May/June: Delivery of Version 1.3. • 17th-18th of July: Delivery of Version 2.0. • + Various demonstrations at NAB, IBC, CES ... Berkom ===!" §
  31. 31. Conclusions • First MPEG-4 applications will appear probably on the Internet based on subsets of MPEG-4 integrated in the IP environment. • In parallel, MPEG-4 can enhance MPEG-2 digital broadcast with attractive and interactive content. • Ongoing AIC-I activity, jointly with W3C, W3D, MPEG-4 will provide specification for advanced interactive content on MPEG-2 and IP environments in an integrated way. Berkom ===!" §
  32. 32. Questions • MPEG : http://www.cselt.it/mpeg (case sensitive) • AIC-I : http://toocan.philabs.research.philips.com/misc/aici/ • Olivier Avaro : o.avaro@berkom.de • Rob Koenen : r.h.koenen@research.kpn.com Berkom ===!" §
  33. 33. Credits • The MPEG-4 Systems Sub-group. • The European project MoMuSys. • Material for the presentation Text and pictures from slides courtesy of KPN Research • Tape demonstration - shows possibilities pictures courtesy of France Telecom CNET, Deutsche Telekom, Instituto Superior Técnico Berkom ===!" §
  34. 34. Annexes Berkom ===!" §
  35. 35. OCI - Object Content Information • Waiting for MPEG-7 would be too long for some application, MPEG-4 designed small, synchronised events containing information descriptors. • Carried in separate ES associated with an Object or in the ES Descriptor of the Object. • Types: – content classification (entity + table number + info - open system! ) – rating, (entity + criteria + info) – language (ISO 639) – textual descriptors (language code + free text) – keywords (ditto) – creation of the content (who, date - no owner! ) Berkom ===!" §
  36. 36. IPMP - Intellectual Property Management and Protection • ‘IPMP’ is prerequisite for ‘publishing’ serious, valuable content in digital form – in any open environment - which nowadays even includes the CD! • IPMP support integrated deeply into the MPEG-4 system • Yet only the minimum is specified – It makes no sense standardizing protection schemes, cryptography , etc. – Only hooks are needed • These hooks are already included in Version 1. Berkom ===!" §
  37. 37. IPMP Architecture Berkom ===!" §

×