MULTIMEDIA AND SYSTEM DESIGN

1. MPEG-4
• Objective
• Standardize algorithms for audiovisual coding in
multimedia applications allowing for
• Interactivity
• High compression
• Scalability of audio and video content
• Support for natural and synthetic audio and video
• The Idea
• An audiovisual scene is a coded representation of
audiovisual objects related in space and time

2. MPEG-4: Scenario
• A/V object
• A video object within a scene
• The background
• An instrument or voice
• Coded independently
• A/V scene
• Mixture of natural or synthetic objects
• Individual bitstreams multiplexed and transmitted
• One or more channels
• Each channel may have its own quality of service

3. MPEG-4: Video Object Plane
• Video frame = sum of segmented regions with
arbitrary shape (VOP)
• Shape motion and texture information of VOPs
belonging to the same video object is encoded into
a video object layer (VOL)
• Encode
• VOL identifiers
• Composition information
• Overlapping configuration of VOPs

4. MPEG-4: Coding
• Shape coding
• Shape information in alpha planes
• Transparency of shape encoded
• Inter and intra shape coding functions
• After shape coding each VOP in a VO is partitioned
into non-overlapping macroblocks
• Motion coding
• Shift parameter wrt reference window
• Standard macroblock
• Contour macroblock

5. MPEG-4: Coding
• Texture coding
• Intra-VOPs, residual errors from motion compensation are DCT
coded like MPEG-1
• 4 luminance and 2 chrominance blocks in a macroblock
• P-VOPs (prediction error blocks) may not conform to VOP
boundary
• Pixels outside the active area are set to a constant value
• Standard compression
• Efficient prediction of DC and AC components from intra and inter
coded blocks
• Multiplexing
• Shape → motion → texture coded data
• Motion and DCT coefficients can be jointly (H.263) or individually
coded

6. MPEG-4 Video Object
Segmentation-I
• Construct a video object
• User selects start frame, outlines polygon designating rough object
boundary
• Refine boundary using snake algorithm, if needed
• Compute a k-pixel bounding box around the object
• Within bounding box compute
• Edge map: bit plane, after thresholding a convolution kernel
• Color map: compute luminance and chrominance, quantize by k-
means clustering, keep quantization table
• Motion field: block-based motion vector
• Segment into regions no significant edge, smooth color having
smooth motion
• Intersect segments and initial object boundary and determine
foreground and background region
• Estimate the motion of regions in the next frame with an affine
motion model

7. MPEG-4 Video Object
Segmentation-II
• Track object
• Locate estimated position of foreground and background regions
from previous frame. Call this the object mask.
• Generate same three feature maps with the quantization table;
Requantize if error is large
• Classify regions into foreground/background and new regions
• Intersection ratio r with object mask
• For foreground regions, if r > 80% OR foreground ∩ mask, mark as
foreground; label foreground - mask as new
• For new regions, if r < 30% mark as new; if r > 80% mark as
foreground; else find nearest-motion-similar neighbor. If it is in the
foreground, do previous step, else keep region as new
• Iterate until stable