Bitmovin LIVE Tech Talks: Fun with Container Formats

The presentation will start in a few minutes...
1

April 20TH
, 2020
Fun with
Container Formats
B I T M O V I N | P R E S E N T A T I O N
All rights reserved. © Bitmovin Inc 2020

Who am I?
Armin Trattnig
Bitmovin Encoding Team
armin.trattnig@bitmovin.com

1. Basics
a. Codecs
b. Terminology
c. Containers
2. MP4
3. MPEG-TS
4. Matroska
5. Adaptive Bitrate Streaming
Agenda

Codecs
Video
H264
HEVC
AV1
Audio
AAC
MP3
OPUS
Subtitles
WebVTT
SRT
SCC

● Elementary Stream
● Media Stream
● Codec Stream
● Video Stream
○ Stream #0:0(eng): Video: h264, 1280x544, 24 fps
● Audio Stream
○ Stream #0:1(ger): Audio: ac3, 48000 Hz, 5.1
○ Stream #0:2(eng): Audio: ac3, 48000 Hz, 5.1
● Subtitle Stream
○ Stream #0:3(ger): Subtitle: subrip
○ Stream #0:4(eng): Subtitle: subrip
Terminology

What is a
media container?

Container format:
Meta file format specification describing how
different multimedia data elements (streams)
and metadata coexist in files
D E F I N I T I O N

What containers provide
Stream encapsulation
Timing/Synchronization
Seeking
Metadata

Common container formats
MP4, fMP4, CMAF, QuickTime, 3GP
MPEG2-TS, MPEG-PS
Matroska, Webm
AV1, MXF, Ogg...

More Terminology
H.264Encoding
H.264Decoding
HEVCH.264Transcoding
H.264Muxing
H.264Demuxing
Transmuxing

MP4

Overview of Standards
MP4: MPEG-4 Part 14
(ISO/IEC 14496-14:2003)
Based on:
ISO base media
ﬁle format
(MPEG-4 Part 12)
Which is based on:
QuickTime File
Format (Apple)
Moving Picture
Experts Group
Coding of
Audio-visual objects

ISO base media ﬁle format
aligned(8) class Box (unsigned int(32) boxtype,
optional unsigned int(8) [16] extended_type) {
unsigned int(32) size;
unsigned int(32) type = boxtype;
If (size==1) {
unsigned int(64) largesize;
} else if (size==0) {
// box extends to end of file
}
if (boxtype==’uuid’) {
unsigned int(8) [16] usertype = extended_type;
}
}
aligned(8) class FileTypeBox
extends Box(‘ftyp’) {
unsigned int(32) major_brand;
unsigned int(32) minor_version;
unsigned int(32) compatible_brands []; // to end of the box

ISO base media ﬁle format
moov
trak
trak
ftyp
meta Other item-specific dataItem info Item location
mdat Item’s Encoded bitstream

Fragmented MP4 (FMP4)
MP4 Container Format
Movie Metadata (moov)
Movie Header (mvhd)
Track
(track)
Movie
Extends
(mvex)
● Track Header (tkhd)
● Media (mdia)
● Movie Extends
Header (mehd)
● Movie Extends (trex)
Fragment
Media Data (mdat)
Media Fragment (moof)
Media
Fragment
Header
(mfhd)
Track
Fragment
(traf)
Fragment
Media Data (mdat)
Media Data (mdat)
Media
Fragment
Header
(mfhd)
Track
Fragment
(traf)

Big Picture of MP4
Streaming on Web
H264, HEVC
MP3, AAC, AC3
Subtitles
Metadata

MPEG-TS

MPEG Transport Stream
● Designed for Digital
Video Broadcasting
(DVB) Applications
● MPEG-TS:
transport-oriented, hence
“transport stream”
● Small individual packets
for tolerance against
transmission errors
● Supports Forward Error
Correction (FEC) for
correcting transmission
errors
● Less metadata than MP4,
e.g. no explicit codec info,
no index of media samples
● More muxing overhead
than MP4

Muxing: ES → PES → TS
ES
Elementary Stream
coming from the encoder
Transport Stream = Sequence of 188 byte long TS packets
Packetized ES
PES Packets
ES
PES
Header
Transport Stream
TS Packets
ES-1 Padding
PES
Header
TS
Header
TS
Header
ES-2
188 bytes 188 bytes
184 bytes

Muxing multiple ES
Each Elementary stream is packetized and muxed into the same Transport Stream.
TS
Muxer
PES PacketizerVideo Encoder
PES PacketizerAudio Encoder
Transport Stream
Audio and Video Packets
interleaved and in temporal
decoding order
Other
Info

Muxing multiple Programs
TS
Muxer
Transport Stream
Containing TS packets
for ES of multiple
Programs
Other
Info
Program 1
Other
Info
Program 2

Muxing multiple Programs
https://en.wikipedia.org/wiki/MPEG_transport_stream All rights reserved. © Bitmovin Inc 2020
Mux
Mux
Audio
Video
Data
Movie
Mux
Audio
Video
Data
News
Cast
Mux
Audio
Video
Data
Sports
Program
Mux
Program Map
Table (PMT)
Electronic Program
Guide (EPG)
More
Data
Table
Transport
stream
TV
ATSC Receiver
Receives the Transport
Stream. Decodes it and
displays on your TV

Matroska (Webm)

Overview
● Free
● Open Source
● Virtually Any Codec
● Extensible Binary
Meta Language (EBML)

Webm
● Based on Matroska
● Google
● VP8, VP9, AV1
● Vorbis, Opus
● HTML-5

Adaptive Bitrate Streaming (DASH, HLS)
Heterogeneous Devices
requesting the right quality for smooth
playback and quick start, no special
server logic needed
Network
with variable
Bandwidth (Internet)
HTTP Server
with video content in
different qualities (e.g.
Apache, IIS or HTTP CDN)

Adaptive Bitrate Streaming
MP4
○ Single init.mp4 file
○ Many small files with media data using mp4 fragments
○ Used by DASH / HLS
MPEG-TS
○ Many small .ts files
○ Used by HLS (Apple)
Matroska
○ Single init file
○ Many small files with media data
○ Very similar to MP4
○ Used by DASH (YouTube/Google)

Compatibility
Problem:
Different platforms support
different container formats
Most browsers:
MPEG-TS
fMP4

MP4
● Online streaming
● Less overhead (compared to MPEG-TS)
● Rich metadata
MPEG-TS
● Broadcasting
● Error correction
● Used by Apple
● Limited metadata
Matroska
● Free
● Any codec
Summary

For you to explore!
https://bitmovin.com/developer-network
https://bitmovin.com/blog/
https://bitmovin.com/container-formats-fun-1/
https://go.bitmovin.com/ultimate-guide-container-formats
L E A R N M O R E

Thank you.

Bitmovin LIVE Tech Talks: Fun with Container Formats

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Bitmovin LIVE Tech Talks: Fun with Container Formats

Similar to Bitmovin LIVE Tech Talks: Fun with Container Formats (20)

More from Bitmovin Inc

More from Bitmovin Inc (19)

Recently uploaded

Recently uploaded (20)

Bitmovin LIVE Tech Talks: Fun with Container Formats