The document provides an overview of the status of MPEG-4 developments and the AIC Initiative. It discusses the goals, history, and architecture of MPEG-4, which aims to code audio-visual objects and scenes to enable interactivity. MPEG-4 extends existing architectures like MPEG-2 and IP to new environments through tools like an interactive scene description and support for new content types and delivery formats. Profiles and levels are defined to suit different applications. Carriage of MPEG-4 over MPEG-2 and IP is also addressed.

1. Overview of MPEG-4
Developments
Olivier Avaro
Chairman MPEG Systems
Deutsche Telekom - Berkom
Davic European Information Day
Geneva, 12 January 1999
Berkom
===!" §

2. Goals of the presentation
• Give a report of the status of MPEG-4 developments.
• Give a report of the status of the AIC Initiative.
• Some special highlights :
– Video and Systems tools,
– Carriage of MPEG-4 content over MPEG-2 and IP.
Berkom
===!" §

3. Messages
• MPEG-4 is not meant to replace existing architectures (ex :
MPEG-2 or IP based architecture).
• MPEG-4 augments existing architecture’s strong points with
new concepts (ex : Interactive Scene Description, new content
types, Java APIs).
• MPEG-4 extends existing architecture’s strong points to new
environments (ex : delivery independent representation of the
content).
Berkom
===!" §

4. Overview
• Introduction to MPEG-4 :
– History, Requirements and Architecture,
– MPEG-4 Video Version 1 tools,
– MPEG-4 Systems Version 1 tools.
• Carriage of MPEG-4 over MPEG-2 and IP.
• Introduction to the AIC Initiative :
– Objectives and architecture,
– Applications,
– Work plan.
• Annexes
– MPEG-4 Object Content Information (OCI)
– MPEG-4 Intellectual Property Management and Protection (IPMP)
Berkom
===!" §

5. A bit of history on MPEG-4
• Participants MPEG: Consumer Electronics, Computer
Industry, Telecom Operators, Academia.
• 1993 : started as ‘Very Low Bitrate Audio-visual Coding’.
• 1994: goal changed to ‘Coding of Audio-visual Objects ’.
• Now: Version 1 is ready; released for IS ballot at the end of
the month. Version 2 released for CD ballot in March.
‘TV/film’
AV-
data
From the MPEG-4
Call for Proposals
(drafted 1994) interactivity
Wireless
‘Computer’
‘Telecom’
Berkom
===!" §

6. Requirements
• Provide common technology for many types of services :
interactive, broadcast, conversational.
• Increase client-side and client-server interactivity.
• Integrate natural and synthetic content.
• Cover a wide range of access conditions.
• Provide serious solutions for the management and
protection of intellectual property.
Berkom
===!" §

7. MPEG-4 Principles
• Audio-visual scenes made of audio-visual objects composed
together according to a scene description :
– allows interaction with elements within the audio-visual scene,
– coding scheme can differ for individual objects,
– allows easy re-use of audio-visual content.
• Audio-visual objects can be of different nature :
– audio (single or multi-channel) or video (arbitrary shape or rectangular),
– natural (natural audio or video) or synthetic (text & graphics, animated faces,
synthetic music),
– 2D (Web like pages) or 3D (spatialized sound, 3D virtual world),
– streamed (video movie) or downloaded (audio jingle).
Berkom
===!" §

8. MPEG-4 Principles
• Scene Description provides :
– the spatial/temporal relationship between the audiovisual objects (2D, 3D,
mixed 2D and 3D scene description),
– the behavior and interactivity of the audio-visual objects and scenes,
– protocols to modify and animate the scene in time,
– a binary encoding for the scene.
• These principles are independent of the bit rate.
Berkom
===!" §

9. Architecture
Decoding
TransMux FlexMux Composition and
N Rendering
Primitive
e
t
AV Objects ...
w ... ...
o
r Ex: MPEG-2 Elementary
Transport Streams Scene Description Audiovisual Interactive Display and User
k Information Scene Interaction
Object Descriptor
Berkom
DAI
===!" §

10. Example of an MPEG-4
Audiovisual Scene (1)
2D Audio-visual scene 2D Audio-visual scene
Audio and Video + Scrolling Text Audio and Video + Still Images
and Still Images
Berkom
===!" §

11. Example of an MPEG-4
Audiovisual Scene (2)
3D Audio-visual scene
3D World + arbitrary shaped
video + still images + 3D
Objects
Berkom
===!" §

12. MPEG-4 Visual Version 1 Tools
• Natural video from 10 kbit/s - 10 Mbit/s
– Rectangular or arbitrary shape objects,
– Scalable,
– Interlaced and Progressive scan supported,
– ‘Sprites’ (e.g. backgrounds): send once, warp after.
• Computer-generated visual information
– facial animation,
– animated meshes with moving texture,
– animated text and graphics.
• All these objects can be synchronised together.
Berkom
===!" §

13. Example of MPEG-4 Video Objects
Arbitrary shape Animated Face
Rectangular shape video object
video object
Berkom
===!" §

14. Visual Version 2 Tools
• Body animation : to augment the face animation tool.
• Compression of 3D meshes with texture :
– games, higher quality virtual environments.
• Higher Quality Video
– currently under discussion.
• Ready end ‘99
• Note that V.2 is a compatible extension of V.1
Berkom
===!" §

15. MPEG-4 Systems Version 1Tools
• Decoder Model
– Predictable behavior of a decoder and decoded content,
– Management of decoder resources (buffer and memory),
– Tight synchronization of Audio, Video and synthetic elements.
• Object Descriptor Protocol
– Identification (location and stream type) of Elementary Streams
(content in the same scene can come from different sources),
– Association of Elementary Streams (alternative content, scalable
content, synchronization hierarchy),
– Association of the scene description and the streams,
– Object Content Information,
– Intellectual property protection and management.
Berkom
===!" §

16. MPEG-4 Systems Version 1Tools
• BIFS Scene Description
– VRML concepts : set of nodes to represent the primitive scene objects to
be composed, the scene graph constructs, the behavior and interactivity
+ Integration of streams,
+ 2D capabilities,
+ Integration of 2D and 3D,
+ Advanced Audio Features,
+ Timing model,
+ BIFS - Update protocol to modify the scene in time,
+ BIFS - Anim protocol to animate the scene in time,
+ Compression efficiency (BIFS is Binary Format for Scene).
Berkom
===!" §

17. Example of MPEG-4 Scenes (1)
2D Audio-visual scene
Audio and Video + Still Images
2D Audio-visual scene
Animated Text+ Video + Still Images
Berkom
===!" §

18. Example of MPEG-4 Scenes (2)
2D and 3D Graphics Primitive
Scene with Face Object
Web-like MPEG-4 Scene
Multi audio and video
MPEG-4 Scene with text
Berkom
===!" §

19. MPEG-4 Systems Version 2 Tools
• File format
– allows interchange and streaming of MPEG-4 content.
• Advanced BIFS
– New BIFS functionality : environmental spatialization, integration of
HTML and BIFS, chroma keying, application signaling,
– Advanced BIFS coding : PROTOs and ExternProtos, Integration of
Mesh, coding of MField.
• MPEG-J
– The MPEG-4 ‘Application Execution Engine’
– MPEG-4 specific API for interaction with the scene (complicated
content behavior), with the network / terminal / UI resources.
Berkom
===!" §

20. MPEG-J Architecture
MPEG-J Run Time Environment
Java MPEG-let
Terminal
Java Virtual Machine
DMIF Application Decoders Compositor
Interface
Berkom
===!" §

21. Profiles and Levels
• Most applications only need a part of the MPEG-4 tool set.
• Profiles define subsets useful for a large class of
applications/services.
• Types of Profiles:
– Scene description (e.g. behaviour),
– Object Descriptor (mainly timing models),
– Audio (natural and synthetic): types of objects,
– Visual (natural and synthetic): types of objects,
– Graphics.
• Levels limit the number of objects and complexity
• Profiles/Levels can be easily added when need arises
Berkom
===!" §

22. Profiles and Levels
MPEG-4 device
Scene
Description
Profiles
Main
speech simple 2D 3D
Audio
Audio
Profiles
core Core
hybrid 3D
FA 2D Object Descriptor
Profile
Visual Graphics
Profiles Profiles
Media Profiles
Berkom
===!" §

23. Carriage of MPEG-4 Content
• Transport of MPEG-4 content : left to other bodies
– Choose IP, MPEG-2 TS, ATM AAL5, …
• DMIF abstracts Application from delivery of data and provide
a uniform walkthrough to access content
– e.g. Broadcast, Internet protocol stack
• Create content once, play on any network configuration
Berkom
===!" §

24. Delivery Framework
SL-Packetized Streams Media
DAI
FlexMux FlexMux
FlexMux Streams
Protection sL Protection sL Delivery
(RTP) (PES) AAL2 H223 ….
UDP MPEG2 ATM PSTN
Mux sub Layer
IP TS
TransMux Streams
Berkom
===!" §

25. MPEG-4 over MPEG-2 and IP
• Technical issues to be addressed :
– Transport/Multiplexing/Packetization (ex : FlexMux, PES only or IETF
RTP multiplex),
– Synchronization (ex : Sync Layer, RTP sync, MPEG-2 sync),
– Identification and association of streams (ex : Object Descriptor, SDP).
• Solved by making tradeoff between :
– How far the integration should be done (ex : using MPEG-2 content or
RTP streams in MPEG-4 scenes, using the MPEG-4 content types in
MPEG-2 or RTP applications),
– How much the duplication of functionality can be reduced.
Berkom
===!" §

26. MPEG-4 over MPEG-2 and IP
• Examples
– Blind transmission of MPEG-4 Programs
• MPEG-4 content separate from others MPEG-2/RTP streams
– MPEG-4 Content (ES) in an MPEG-2 Systems /RTP application
• adding MPEG-4 content types to MPEG-2/RTP
– MPEG-4 ES + Systems in MPEG-2/RTP environment
• Adding MPEG-4 Interactive programming to MPEG-2/RTP
• MPEG-2/RTP ‘Objects’ may be used in the MPEG-4 application
• On-going activity within MPEG and IETF that should lead for
end 99 to :
– addenda to MPEG-2 and MPEG-4,
– IETF RFC.
Berkom
===!" §

27. AIC-I Charter
• Goal: Harmonize standards efforts and enable
interactive 2D, 3D and streaming content for commerce
and entertainment on set top devices.
• The Advanced Interactive Content Initiative (AIC Initiative)
applied to be an industry consortium operating under the
Industry Technical Agreement (ITA) of the International
Electro technical Commission (IEC).
• The purpose of the AIC Initiative is the development of
interactive content specifications for user devices to offer a
range of basic through advanced interactive applications using
3D as well as 2D content, in stored and streamed form.
Berkom
===!" §

28. AIC-I Applications
• Advanced Electronic Program Guide.
• Buy-me-button.
• Data Ticker.
• Program Enhancement.
• Interactive Home Shopping.
• Interactive Commercial.
• Interactive Entertainment.
• Demographic Programming.
Berkom
===!" §

29. AIC-I Principles
• Delivery in various transport environments
– including MPEG-2 and IP environments,
– and combinations of broadcast/interactive delivery.
• Presentation engine :
– One specification for integration of MPEG-4, ‘X-HTML’, VRML
content.
• Application execution engine
– Not the priority right now.
Berkom
===!" §

30. AIC-I Workplan
• 20th December: Delivery of Version 1.0.
• 8th-9th February: Delivery of Version 1.1.
• 21st-22nd of March: Field trial #1. Delivery of Version 1.2.
• May/June: Delivery of Version 1.3.
• 17th-18th of July: Delivery of Version 2.0.
• + Various demonstrations at NAB, IBC, CES ...
Berkom
===!" §

31. Conclusions
• First MPEG-4 applications will appear probably on the Internet
based on subsets of MPEG-4 integrated in the IP environment.
• In parallel, MPEG-4 can enhance MPEG-2 digital broadcast
with attractive and interactive content.
• Ongoing AIC-I activity, jointly with W3C, W3D, MPEG-4
will provide specification for advanced interactive content on
MPEG-2 and IP environments in an integrated way.
Berkom
===!" §

32. Questions
• MPEG : http://www.cselt.it/mpeg (case sensitive)
• AIC-I : http://toocan.philabs.research.philips.com/misc/aici/
• Olivier Avaro : o.avaro@berkom.de
• Rob Koenen : r.h.koenen@research.kpn.com
Berkom
===!" §

33. Credits
• The MPEG-4 Systems Sub-group.
• The European project MoMuSys.
• Material for the presentation
Text and pictures from slides courtesy of KPN Research
• Tape demonstration - shows possibilities
pictures courtesy of France Telecom CNET, Deutsche
Telekom, Instituto Superior Técnico
Berkom
===!" §

34. Annexes
Berkom
===!" §

35. OCI - Object Content Information
• Waiting for MPEG-7 would be too long for some application,
MPEG-4 designed small, synchronised events containing
information descriptors.
• Carried in separate ES associated with an Object or in the ES
Descriptor of the Object.
• Types:
– content classification (entity + table number + info - open system! )
– rating, (entity + criteria + info)
– language (ISO 639)
– textual descriptors (language code + free text)
– keywords (ditto)
– creation of the content (who, date - no owner! )
Berkom
===!" §

36. IPMP - Intellectual Property
Management and Protection
• ‘IPMP’ is prerequisite for ‘publishing’ serious, valuable
content in digital form
– in any open environment - which nowadays even includes the CD!
• IPMP support integrated deeply into the MPEG-4 system
• Yet only the minimum is specified
– It makes no sense standardizing protection schemes, cryptography ,
etc.
– Only hooks are needed
• These hooks are already included in Version 1.
Berkom
===!" §

37. IPMP Architecture
Berkom
===!" §