Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Dynamic	Adaptive	Streaming	over	HTTP
Overview,	State	of	the	Art,	and	Challenges
Priv.-Doz.	Dr.	Christian	Timmerer
[Ack:	Al...
ACM	MMSys 2018	CfP is	Out!
• City:	Amsterdam
• Dates:	June	12-15,	2018
• Co-located	with	
– NOSSDAV
– MoVid
– MMVE
• http:...
July	14,	2017 Dr.	Timmerer	[AAU/Bitmovin] 3
Importance	of	Multimedia	Delivery
• Multimedia	is	predominant	on	the	Internet
• Real-time	entertainment:	Streaming	video	a...
Open	Digital	Media	Value	Chain
Create	
Content
Aggregate
Monetize
Distribute	
Content
Consume	
Content
Any	Content Any	Sto...
Example	Platform/Infrastructure
https://bitmovin.com/
July	14,	2017 Dr.	Timmerer	[AAU/Bitmovin] 6
Common	Annoyances	in	Streaming
• Wrong	format
• Wrong	protocol
• Plugin	requirements
• DRM	issues
• Long	start-up	delay
• ...
What	is	DASH?
July	14,	2017 Dr.	Timmerer	[AAU/Bitmovin] 8
Reading: http://en.wikipedia.org/wiki/Dash_(disambiguation)
Over-The-Top	– Adaptive	Media	Streaming
• In	a	nutshell…
Adaptation logic is within the
client, not normatively
specified ...
Multi-Bitrate	Encoding	and	Representation	Switching
Contents	on	the	Web	Server
Movie	A	– 200	Kbps
Movie	A	– 400	Kbps
Movie...
Adaptive	Streaming	over	HTTP
…
…
…
…
HTTP GETs
Client
Buffer
Media
Player
HTTP
Server
July	14,	2017 Dr.	Timmerer	[AAU/Bitm...
Formats	and	Standards
• Adobe
– HTTP	Dynamic	Streaming	(HDS)
– Switched	to	DASH
• Apple
– HTTP	Live	Streaming	(HLS)
– Requ...
Scope	of	DASH:	what	is	specified?
Media Presentation on
HTTP Server
DASH-enabled ClientMedia Presentation
Description
.
.
...
Scope	of	DASH:	what	is	specified?
Media Presentation on
HTTP Server
DASH-enabled ClientMedia Presentation
Description
.
.
...
DASH	Data	Model
MPD
Period	id	=	1
start	=	0	s
Period	id	=	3
start	=	300	s
Period	id	=	4
start	=	850	s
Period	id	=	2
start	...
Profile	identifier
“static”	|	“dynamic”
Multiple	content	
locations
Time	sequence	of	Media	
Presentation
Client	(QoE)	metr...
MPD	Schema	Overview
Audio/Video	
parameters
Container,	codec,	
information
Bandwidth
Quality	information
Descriptors
URL	c...
Segment Index in MPD only
Segment Index in MPD + Segment
Segment Index in Segment only
<MPD>
...
<URL sourceURL="seg1.m4s"...
type=static typically,
for on demand content
Base URL of the
segments
Subtitles
Audio adaptation set
with different
repres...
https://bitmovin.com/demo/
July	14,	2017 Dr.	Timmerer	[AAU/Bitmovin] 20
ISO/IEC	23009-1	Timeline
Fastest	time	ever	that	a	standard	was	developed	in	MPEG	to	address	the	demand	of	
the	market
• Ot...
MPEG-DASH	Features	(Jan‘17)
July	14,	2017 Dr.	Timmerer	[AAU/Bitmovin] 22
Ongoing	Work	in	MPEG-DASH
• Currently	Running	Core	Experiments
– High	Quality	VR	delivery	with	DASH	(DASH-VR)	
https://bit...
MPEG	CMAF:	Threat	or	Opportunity?
+ CMAF	Media	Object	
Model	compatible	with	
DASH	Data	Model
+ Segment	formats	based	
on	...
CMAF	Development	Timeline
Feb.	2016	
Working	Draft
June	2016	
Committee	Draft
Nov.	2016	
Draft	
International	
Standard
Ja...
CMAF	ISO-BMFF	Media	Objects
• Manifests	typically	provide	URLs	to
– CMAF	track	files
– CMAF	header	+	CMAF	segments
• singl...
CMAF	Chunks
CMAF	
Header
CMAF	Content	Consumption
Un-
packaging
MSE
CMAF	
Track	File
CMAF	Fragments
Switching	can	only	hap...
Common	Problems	in	DASH
• Encoding |	Packaging |	Encryption
– Guidelines:	3-20	different	representations	(mobile	to	UHD)
–...
Quality	of Experience	for DASH
• Objective
– Initial	or	startup	delay	(low)	
– Buffer	underrun	/	stalls	(zero)
– Quality	s...
DASH	QoE	in	Real-World
B. Rainer, C. Timmerer, “Quality of Experience of Web-based Adaptive HTTP
Streaming Clients in Real...
10	different	
adaptation	
algorithms
Christian Timmerer, Matteo Maiero,
Benjamin Rainer, Which Adaptation Logic?
An Object...
July	14,	2017 32
July	14,	2017 Dr.	Timmerer	[AAU/Bitmovin] 33
Comparison	of	eight	DASH	Players
Conclusions
• MPEG-DASH	defines	formats	only
– Media	Presentation	Description	(MPD)
– Segment	format:	isobmff,	m2ts
• MPEG...
Deployment	Thoughts
• Proprietary	ecosystems	will	disappear	(Silverlight,	Flash)
• No	more	plugins	– HTML5!
– MSE/EME	avai...
Upcoming SlideShare
Loading in …5
×

MPEG-DASH: Overview, State-of-the-Art, and Future Roadmap

3,320 views

Published on

This presentation provides an overview of MPEG-DASH and future developments, namely common media application format and virtual reality/360-degree streaming.

Published in: Technology

MPEG-DASH: Overview, State-of-the-Art, and Future Roadmap

  1. 1. Dynamic Adaptive Streaming over HTTP Overview, State of the Art, and Challenges Priv.-Doz. Dr. Christian Timmerer [Ack: Ali C. Begen, Networked Media & Ozyegin University | Thomas Stockhammer, Qualcomm Inc. Iraj Sodagar, Microsoft | C. Concolato, Netflix] Alpen-Adria-Universität Klagenfurt (AAU) w Multimedia Communication (MMC) http://blog.timmerer.com w christian.timmerer@itec.aau.at Chief Innovation Officer (CIO) at bitmovin GmbH http://www.bitmovin.com w christian.timmerer@bitmovin.com Mobile Multimedia Computing @ IEEE ICME 2017, Hong Kong, Jul 14, 2017 http://www.slideshare.net/christian.timmerer Source: Wikipedia Change Log: • EBU/ASBU Week of Technology, Tunis, Oct 19, 2016 • TU Wien, Vienna, Feb 9, 2017 • TNC17, Linz, May 30, 2017 • MMC, IEEE ICME’17, Hong Kong, Jul 14, 2017
  2. 2. ACM MMSys 2018 CfP is Out! • City: Amsterdam • Dates: June 12-15, 2018 • Co-located with – NOSSDAV – MoVid – MMVE • http://www.mmsys2018.org/ July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 2
  3. 3. July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 3
  4. 4. Importance of Multimedia Delivery • Multimedia is predominant on the Internet • Real-time entertainment: Streaming video and audio; >70% of Internet traffic at peak periods • Popular services: YouTube (17.53%), Netflix (35.15%), Amazon Video (4.26%), Hulu (2.68%); all delivered over-the-top (OTT); huge potential for mobile access! Global Internet Phenomena Report: 2016 July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 4
  5. 5. Open Digital Media Value Chain Create Content Aggregate Monetize Distribute Content Consume Content Any Content Any Storefront Any Network Any Device CDNsMedia Protocols Internet Transport DRM Encoding Encapsulation Dynamic Ads Clients Happy User July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 5
  6. 6. Example Platform/Infrastructure https://bitmovin.com/ July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 6
  7. 7. Common Annoyances in Streaming • Wrong format • Wrong protocol • Plugin requirements • DRM issues • Long start-up delay • Poor quality • Frequent stalls • Quality oscillations • No seeking features July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 7
  8. 8. What is DASH? July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 8 Reading: http://en.wikipedia.org/wiki/Dash_(disambiguation)
  9. 9. Over-The-Top – Adaptive Media Streaming • In a nutshell… Adaptation logic is within the client, not normatively specified by the standard, subject to research and development July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 9
  10. 10. Multi-Bitrate Encoding and Representation Switching Contents on the Web Server Movie A – 200 Kbps Movie A – 400 Kbps Movie A – 1.2 Mbps Movie A – 2.2 Mbps . . . . . . Movie K – 200 Kbps Movie K – 500 Kbps Movie K – 1.1 Mbps Movie K – 1.8 Mbps . . . . . . Time (s) Start quickly Keep requesting Improve quality Loss/congestion detection Revamp quality . . . . . . Segments July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 10
  11. 11. Adaptive Streaming over HTTP … … … … HTTP GETs Client Buffer Media Player HTTP Server July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 11
  12. 12. Formats and Standards • Adobe – HTTP Dynamic Streaming (HDS) – Switched to DASH • Apple – HTTP Live Streaming (HLS) – Required for iOS • Microsoft – Smooth Streaming – Switched to DASH, almost.. • MPEG Dynamic Adaptive Streaming over HTTP (DASH) – Supported by Netflix, YouTube, Bitmovin, etc. • MPEG Common Media Application Format (MPEG-A Part 19) – The new kid on the block – support for “fragmented mp4 in HLS” – DASH/HLS convergence at segment level – some open issues with encryption format Source: http://xkcd.com/927/ July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 12
  13. 13. Scope of DASH: what is specified? Media Presentation on HTTP Server DASH-enabled ClientMedia Presentation Description . . . Segment … . . .Segment … . . . Segment … . . .Segment … … Segments located by HTTP-URLs DASH Control Engine HTTP/1.1 HTTP Client MPD Parser Media Engine On-time HTTP requests to segments July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 13
  14. 14. Scope of DASH: what is specified? Media Presentation on HTTP Server DASH-enabled ClientMedia Presentation Description . . . Segment … . . .Segment … . . . Segment … . . .Segment … … Segments located by HTTP-URLs DASH Control Engine HTTP/1.1 HTTP Client MPD Parser Media Engine On-time HTTP requests to segments July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 14
  15. 15. DASH Data Model MPD Period id = 1 start = 0 s Period id = 3 start = 300 s Period id = 4 start = 850 s Period id = 2 start = 100 s Adaptation Set 0 subtitle turkish Adaptation Set 2 audio english Adaptation Set 1 BaseURL=http://abr.rocks.com/ Representation 2 Rate = 1 Mbps Representation 4 Rate = 3 Mbps Representation 1 Rate = 500 Kbps Representation 3 Rate = 2 Mbps Resolution = 720p Segment Info Duration = 10 s Template: 3/$Number$.mp4 Segment Access Initialization Segment http://abr.rocks.com/3/0.mp4 Media Segment 1 start = 0 s http://abr.rocks.com/3/1.mp4 Media Segment 2 start = 10 s http://abr.rocks.com/3/2.mp4 Adaptation Set 3 audio german Adaptation Set 1 video Period id = 2 start = 100 s Representation 3 Rate = 2 Mbps Selection of components/tracks Well-defined media format Selection of representations Splicing of arbitrary content like ads Chunks with addresses and timing July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 15
  16. 16. Profile identifier “static” | “dynamic” Multiple content locations Time sequence of Media Presentation Client (QoE) metrics Set of switchable Representations Encoded version of a media component MPD Schema Overview July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 16
  17. 17. MPD Schema Overview Audio/Video parameters Container, codec, information Bandwidth Quality information Descriptors URL construction Playlist-based Template-based July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 17
  18. 18. Segment Index in MPD only Segment Index in MPD + Segment Segment Index in Segment only <MPD> ... <URL sourceURL="seg1.m4s"/> <URL sourceURL="seg2.m4s"/> </MPD> seg1.m4s seg2.m4s ... <MPD> ... <URL sourceURL="seg.m4s" range="0-499"/> <URL sourceURL="seg.m4s" range="500-999"/> </MPD> seg.m4s <MPD> ... <Index sourceURL="sidx.mp4"/> <URL sourceURL="seg.m4s"/> </MPD> seg.m4s sidx. m4s <MPD> ... <BaseURL>seg.m4s</BaseURL> </MPD> seg.m4ssidx July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 18
  19. 19. type=static typically, for on demand content Base URL of the segments Subtitles Audio adaptation set with different representations (bw) Video adaptation set with different representations (bw) Different codecs (profiles) Segment URL constructed with template and base URL July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 19
  20. 20. https://bitmovin.com/demo/ July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 20
  21. 21. ISO/IEC 23009-1 Timeline Fastest time ever that a standard was developed in MPEG to address the demand of the market • Other Relevant Specifications – 14496-12: ISO Base Media File Format – 14496-15: Carriage of NAL unit structured video in the ISO Base Media File Format – 23001-7: Common encryption format for ISO base media file format – 23001-8: Coding-Independent Code Points – 23001-10: Carriage of Timed Metadata Metrics of Media in ISO Base Media File Format CfP Issued April 2010 18 Responses and Working Draft (WD) July 2010 Committee Draft (CD) Oct. 2010 Draft International Standard (DIS) Jan. 2011 Final Draft International Standard Nov. 2011 Published as International Standard April 2012 See also here https://multimediacommunication.blogspot.co.at/2010/05/http-streaming-of-mpeg-media.html July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 21
  22. 22. MPEG-DASH Features (Jan‘17) July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 22
  23. 23. Ongoing Work in MPEG-DASH • Currently Running Core Experiments – High Quality VR delivery with DASH (DASH-VR) https://bitmovin.com/mpeg-vr-one-ring-rule/ https://bitmovin.com/lighter-faster-interactive-ads-cross-platform-vr- bitmovins-latest-major-player-version-6-0/ New work item proposal Coded Representation of Immersive Media July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 23
  24. 24. MPEG CMAF: Threat or Opportunity? + CMAF Media Object Model compatible with DASH Data Model + Segment formats based on ISOBMFF - Different manifest formats (MPD vs. m3u8) - CENC: AES-128 CBC (HLS) vs. AES-128 CTR (all others) mode https://bitmovin.com/what-is-cmaf-threat-opportunity/ July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 24
  25. 25. CMAF Development Timeline Feb. 2016 Working Draft June 2016 Committee Draft Nov. 2016 Draft International Standard Jan. 2017 Study of Draft International Standard May 2017 Final Draft International Standard July 2017 Approved International Standard Ballot Ballot Ballot July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 25
  26. 26. CMAF ISO-BMFF Media Objects • Manifests typically provide URLs to – CMAF track files – CMAF header + CMAF segments • single/multiple fragment(s) – CMAF header + CMAF chunk Encoder Encryption Packaging CMAF Header CMAF Fragment CMAF Fragment CMAF Chunk CMAF Chunk CMAF Chunk CMAF Fragment R A P R A P R A P R A P CMAF Fragment CMAF Segment CMAF Segment CMAF Track File July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 26
  27. 27. CMAF Chunks CMAF Header CMAF Content Consumption Un- packaging MSE CMAF Track File CMAF Fragments Switching can only happen seamlessly at CMAF Fragment boundaries CMAF Segments CMAF Header CMAF Header July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 27
  28. 28. Common Problems in DASH • Encoding | Packaging | Encryption – Guidelines: 3-20 different representations (mobile to UHD) – Segment length: 4s shows good tradeoff (2s vs. 9s) • https://bitmovin.com/mpeg-dash-hls-segment-length/ – Offline vs. on-the-fly • Delivery, distribution, CDN – MMSys’16 keynote by Neill Kipp: https://mmsys2016.itec.aau.at/ • Consumption and Quality of Experience (QoE) – Adaptation strategies: buffer- vs. throughput-based – Multi-client competition: on-off behavior – Quality-aware streaming: highest possible bitrate vs. highest quality – Inter-Destination Media Synchronization (IDMS): new applications – Virtual Reality / 360-degree video: tiled streaming July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 28
  29. 29. Quality of Experience for DASH • Objective – Initial or startup delay (low) – Buffer underrun / stalls (zero) – Quality switches (low) – Media throughput (high) – [Other media-related configuration: encoding, representations, segment length, …] • Subjective – Mean Opinion Score (MOS) – various scales – Various methodologies (e.g., DSCQS, DSIS, ACR, PC, …) M. Seufert, et al., "A Survey on Quality of Experience of HTTP Adaptive Streaming,"IEEE Communications Surveys & Tutorials, vol.17, no.1, 2015. doi:10.1109/COMST.2014.2360940 July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 29
  30. 30. DASH QoE in Real-World B. Rainer, C. Timmerer, “Quality of Experience of Web-based Adaptive HTTP Streaming Clients in Real-World Environments using Crowdsourcing”, Proceedings of International Workshop on VideoNext: Design, Quality and Deployment of Adaptive Video Streaming, Sydney, Australia, Dec. 2014. Stalls and low quality are bad for QoE but not startup delay July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 30
  31. 31. 10 different adaptation algorithms Christian Timmerer, Matteo Maiero, Benjamin Rainer, Which Adaptation Logic? An Objective and Subjective Performance Evaluation of HTTP-based Adaptive Media Streaming Systems, In arXiv.org [cs.MM], N.N., vol. abs/1606.00341, N.N., pp. 11, 2016. July 14, 2017 31
  32. 32. July 14, 2017 32
  33. 33. July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 33 Comparison of eight DASH Players
  34. 34. Conclusions • MPEG-DASH defines formats only – Media Presentation Description (MPD) – Segment format: isobmff, m2ts • MPEG-DASH is not – System, protocol, presentation, codec, interactivity, DRM, client specification • Other standards required for a complete ecosystem – DASH-IF, WAVE, HMTL5, MSE, EME, 3GPP, DVB, etc. • Do we need MPEG-DASH? (for adaptive media streaming) – Not necessarily: e.g., WebM + VPx + manifest & control end-to-end – Required to address heterogeneous environments to solve interop. problem • Role of standards sometimes overrated but often underestimated • CMAF and VR addressing new challenges for adaptive media streaming July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 34
  35. 35. Deployment Thoughts • Proprietary ecosystems will disappear (Silverlight, Flash) • No more plugins – HTML5! – MSE/EME available on all major browser platforms – Support for both DASH/HLS (+CMAF) and CENC • Rich feature set: codecs, ads, DRM, multi-language/-audio, subtitles, VR/360, UHD, HFR, HDR, live, on-demand, analytics, … • Common implementation issues: start-up, buffering, high-quality, seamless switching, platform support, cost-effective, … • Solutions available for adaptive streaming, advertising, VR/360, live streaming, and DRM – Details available at https://bitmovin.com/ July 14, 2017 Dr. Timmerer [AAU/Bitmovin] 35

×