SlideShare a Scribd company logo
1 of 47
Download to read offline
Media Source Extensions 
Streaming Video without Plugins 
Jeff Tapper 
Digital Primates 
@jefftapper
Who am I? 
• Senior Consultant at Digital Primates 
– Building next generation client applications 
• Built video applications for many of the most 
watched live broadcasts 
• Developing Internet applications for 19 years 
• Author of 12 books on Internet technologies
Who are you?
Agenda 
• Video and the Internet today 
• Understanding HTTP Streaming 
• What are the Streaming options without a 
plugin? 
• What is DASH 
• What is DASH-264 
• Making it work in a browser 
• Questions
Online video Options 
• Progressive Download 
• Real Time Protocols (RTP, RTMP, RTSP, etc) 
• HTTP Streaming (HDS, HLS, Smooth 
Streaming, etc)
The challenge 
• Most agree that HTTP Streaming is the most 
efficient choice 
• Different devices support different streaming 
protocols 
• No one standard is currently supported 
ubiquitously 
• Results in media being served in several 
different formats to support the broadest 
range of devices
What do browsers support? 
• Unfortunately, Progressive Download is the only 
ubiquitously supported option 
• Different Browsers support different video 
codec’s 
– H.264 
– webM 
– VP8/VP9 
– Etc. 
• Safari (iOs and MacOS only) natively supports HLS 
• MediaSource Extensions released in Chrome and 
IE11, betas in Safari and Firefox
MediaSource Extensions (MSE) 
• MSE allow for pieces (segments) of media to 
be handed to the HTML5 video tag’s buffer 
directly. 
• This enables HTTP Streaming in HTML 
• Not universally supported, yet. 
• Currently (as of September 2014) an Editors 
Draft to the HTML Working Group
What is MPEG-DASH 
 DASH – Dynamic Adaptive Streaming via HTTP 
 International open standard, developed and 
published by ISO 
 Addresses both simple and advanced use cases 
 Enables highest-quality multiscreen distribution 
and efficient dynamic adaptive switching 
 Enables reuse of existing content, devices and 
infrastructure 
 Attempts to unify to a single standard for HTTP 
Streaming
DASH and codecs 
• The DASH specification is codec agnostic 
• Any existing or future codec can work with 
DASH 
• DASH manifest describes which codec is used 
• Allows ability for a single manifest to describe 
several different versions in different codecs
DASH264 
• H.264 is dominant format today 
• Many vendors and service providers are 
committed to supporting/enabling DASH264 
• Provides support for today’s requirements 
such as DRM 
• H.264 is backed by rigorous testing and 
conformance
DASH Industry Forum 
• Addressing the dramatic growth of broadband 
video by recommending a universal delivery 
format that provides end users with the best 
possible media experience by dynamically 
adapting to changing network conditions.
DASH Industry Forum 
• Objectives: 
– promote and catalyze market adoption of MPEG-DASH 
– publish interoperability and deployment guidelines 
– facilitate interoperability tests 
– collaborate with standard bodies and industry 
consortia in aligning ongoing DASH standards 
development and the use of common profiles across 
industry organizations 
• Over 65 members 
• Visit http://dashif.org for more information 
• Released the DASH/264 standard
Building a DASH player 
• We have built DASH players for several 
different platforms 
– Flash 
– Android 
– HTML5/JavaScript (dash.js) 
• DASH.js is available as an open source project 
(bsd3) on github 
• DASH.js is the reference player for the DASH 
Industry Forum (dashif.org)
How to play a DASH Stream 
• Download Manifest 
• Parse Manifest 
• Determine optimal bandwidth for client 
• Initialize for bandwidth 
• Download Segment 
• Hand segment to MSE 
• Check Bandwidth to determine if change is 
necessary
Understanding DASH structure 
• Three types of files 
– Manifest (.mpd) 
• XML file describing the segments 
– Initialization file 
• Contains headers needed to decode bytes in segments 
– Segment Files 
• Contains playable media 
• Includes: 
– 0…many video tracks 
– 0…many audio tracks
DASH Manifest 
• Manifest contains: 
– Root node 
– 1 or more periods 
• Periods contain 1 adaptation set per video stream and 
• Periods contain 1 adaptation set per audio stream 
• Adaptation Sets contain: 
– Content Composition nodes (for each video or audio track) 
– 1 or more Representation node 
» Each representation describes a single bitrate 
» Representations contain data on finding the actual segments 
» Different ways a representation can describe segments
Describing Representations 
• SegmentBase 
– Describes a stream with only a single Segment per bitrate 
– Can be used for Byte Range Requests 
• SegmentList 
– A SegmentList will contain a specific list of each 
SegmentURL (individual HTTP packet with media data) 
– Can be used for Byte Range Requests 
• SegmentTemplate 
– Defines a known url for the fragment with wildcards 
resolved at runtime to request a segments (see bbb.mpd) 
– Alternatively, can specify a list of segments based on 
duration
SegmentList 
<Representation id="h264bl_hd" mimeType="video/mp4" 
codecs="avc1.42c01f" width="1280" height="720" startWithSAP="1" 
bandwidth="514864"> 
<SegmentList timescale="1000" duration="10000"> 
<Initialization sourceURL="mp4-main-multi-h264bl_hd-.mp4"/> 
<SegmentURL media="mp4-main-multi-h264bl_hd-1.m4s"/> 
<SegmentURL media="mp4-main-multi-h264bl_hd-2.m4s"/> 
<SegmentURL media="mp4-main-multi-h264bl_hd-3.m4s"/> 
<SegmentURL media="mp4-main-multi-h264bl_hd-4.m4s"/> 
<SegmentURL media="mp4-main-multi-h264bl_hd-5.m4s"/> 
<SegmentURL media="mp4-main-multi-h264bl_hd-6.m4s"/> 
<SegmentURL media="mp4-main-multi-h264bl_hd-7.m4s"/> 
<SegmentURL media="mp4-main-multi-h264bl_hd-8.m4s"/>
SegmentTemplate fixed segment 
duration 
<AdaptationSet> 
<ContentComponent id="1" contentType="video"/> 
<SegmentTemplate 
initialization="BigBuckBunny_720p_1800kbps_44khz_track1_dash.mp4"/> 
<Representation id="1" 
mimeType="video/mp4“ codecs="avc1.64001f" 
width="1280" height="720“ 
startWithSAP="1" bandwidth="1809954"> 
<SegmentTemplate timescale="1000" duration="13809" 
media="bbb_seg_BigBuckBunny_720p_1800kbps_44khz_track1$Number$.m4s" 
startNumber="1"/> 
</Representation> 
</AdaptationSet>
SegmentTemplate variable segment 
duration 
<AdaptationSet group="2" mimeType="video/mp4" par="16:9“ minBandwidth="475000“ 
maxBandwidth="6589000" minWidth="176" maxWidth="1680" 
minHeight="99" maxHeight="944“ segmentAlignment="true“ 
startWithSAP="1"> 
<SegmentTemplate timescale="1000" 
initialization="dash/ateam-video=$Bandwidth$.dash" 
media="dash/ateam-video=$Bandwidth$-$Time$.dash"> 
<SegmentTimeline> 
<S t="0" d="4171" /> 
<S d="2503" /> 
<S d="2961" /> 
<S d="2461" /> 
<S d="2127" r="2" /> 
…
dash.js player
dash.js player 
• dash.js is a free open source player 
• Code available on github 
• Currently the base of several different 
production players 
• Recent uses include: 
– BBC live broadcasts 
– Wowza 
– EZDRM 
– And more!
How to play a DASH Stream 
• Download Manifest 
• Parse Manifest 
• Determine optimal bandwidth for client 
• Initialize for bandwidth 
• Download Segment 
• Hand segment to MSE 
• Check Bandwidth to determine if change is 
necessary
Tools used by dash.js 
Core Player 
• Dijon – DI / IOC 
Development 
• Jasmine – unit tests 
Web Site 
• AngularJS – Application Framework 
• Flat-ui – UI elements 
• Flot – Charting 
• Kendo - Components
Class Structure 
• The player is divided into two main packages. 
• streaming – Contains the classes responsible 
for creating and populating the MediaSource 
buffers. These classes are intended to be 
abstract enough for use with any segmented 
stream (such as DASH, HLS, HDS and MSS). 
• dash – Contains the classes responsible for 
making decisions specifically related to Dash.
streaming package
MediaPlayer.js 
• Exposes the top level functions and properties 
to the developer (play, autoPlay, isLive, abr 
quality, and metrics). 
• The manifest URL and the HTML Video object 
as passed to the MediaPlayer.
Context.js 
• The dependency mapping for the stream 
package. 
• The context is passed into the MediaPlayer 
object allowing for different MediaPlayer 
instances to use different mappings.
Stream.js 
• Loads/refreshes the manifest. 
• Create SourceBuffers from MediaSource. 
• Create BufferManager classes to manage 
SourceBuffers. 
• Responds to events from HTML Video object. 
• For a live stream, the live edge is calculated 
and passed to the BufferController instances.
Debug.js 
• Convenience class for logging methods. 
• Default implementation is to just use 
console.log(). 
• Extension point for tapping into logging 
messages.
BufferController.js 
• Responsible for loading fragments and 
pushing the bytes into the SourceBuffer. 
• Once play() has been called a timer is 
started to check the status of the bytes in the 
buffer. 
• If the amount of time left to play is less than 
Manifest.minBufferTime the next fragment 
is loaded. 
• Records metrics related to playback.
ManifestLoader.js 
• Responsible for loading manifest files. 
• Returns the parsed manifest object. 
FragmentLoader.js 
• Responsible for loading fragments. 
• Loads requests sequentially.
AbrController.js 
• Responsible for deciding if the current quality 
should be changed. 
• The stream metrics are passed to a set of 
‘rules’. 
• Methods: 
getPlaybackQuality(type, data) 
 type – The type of the data 
(audio/video). 
 data – The stream data.
DownloadRatioRule.js 
• Validates that fragments are being 
downloaded in a timely manner. 
• Compares the time it takes to download a 
fragment to how long it takes to play out a 
fragment. 
• If the download time is considered a 
bottleneck the quality will be lowered.
InsufficientBufferRule.js 
• Validates that the buffer doesn’t run dry 
during playback. 
• If the buffer is running dry continuously it 
likely means that the player has a processing 
bottleneck (video decode time is longer than 
playback time).
dash package
DashContext.js 
• Defines dependency mapping specific to the 
dash package. 
– Parser 
– Index Handler 
– Manifest Extensions
DashParser.js 
• Converts the manifest to a JSON object. 
• Converts duration and datetime strings into 
number/date objects. 
• Manages inheritance fields. 
– Many fields are inherited from parent to child 
nodes in DASH. 
– For example, a BaseURL can be defined in the 
<MPD> node and all <Representation> nodes 
inherit that value.
DashHandler.js 
• Responsible for deciding which fragment URL should be 
loaded. 
• Methods: 
 getInitRequest(quality) – Returns an initialization 
request for a given quality, if available. 
 getSegmentRequestForTime(time, quality) – Returns 
a fragment URL to load for a given quality and a given 
time. Returns a Stream.vo.SegmentRequest object. 
 getNextSegmentRequest(quality) – Returns the next 
fragment URL to load. Assumes that 
getSegmentRequestForTime() has already been called. 
 getCurrentTime (quality) – Returns the time for the 
last loaded fragment index.
DashHandler.js (cont’d) 
• Uses available information in the manifest (SegmentList, 
SegmentTemplate, SegmentBase). 
• When using a single, non-fragmented mp4 file the SIDX box 
will be loaded to determine byte ranges for segments.
Flow 
1. Create the Context and MediaPlayer instances. 
var context = new Dash.di.DashContext(), 
player = new MediaPlayer(context); 
2. Initialize MediaPlayer and set manifest URL. 
player.startup(); 
player.setIsLive(false); 
player.attachSource(manifest_url); 
3. Attach HTML Video element. 
video = document.querySelector(".dash-video-player 
video"), 
player.autoPlay = true; 
player.attachView(video);
2. Call play()on the MediaPlayer (if autoPlay = 
false). 
3. The Stream object will be created and initialized with the 
manifest URL. 
4. The manifest is loaded and then parsed. 
5.MediaSource, SourceBuffers, and 
BufferControllers are created. 
– Create one BufferController per stream type (usually 
video and audio). 
6. Set the duration of the MediaSource to the duration of the 
manifest (or infinity for a live stream). 
7. If the stream is live, calculate the live edge. 
8. Call play() on the HTML video element. 
9. The BufferManager instances create a timer. When the 
timer ticks the state of the buffers is checked.
BufferManager.validate() 
1. Check to see if the buffers need more data. 
• Must be in a playing state. 
• Must not already be loading data. 
• Must require more data to be buffered. 
amountBuffered < manifest.minBufferTime 
2. If automatic ABR is enabled check to see if the bitrate 
should be changed. 
• Ask AbrController for the new quality. 
• Rules will determine which bitrate to change to. 
3. If initial playback, seeking, or the bitrate has changed load 
the initialization fragment (if available).
4. Ask the IndexHandler for the next fragment request. 
• If seeking pass the seek time to the IndexHandler. 
• Otherwise ask for the ‘next’ fragment. 
• Pass the bitrate to the IndexHandler. 
6. The IndexHandler returns a SegmentRequest indicating 
what action the BufferManager should take next. 
• “download” – Download and append the fragment to the buffer. 
• “stall” – Wait because the IndexHandler is not ready. 
• “complete” – Signal that the stream has completed playback. 
7. Repeat.
Resources 
• DASH Industry Forum 
– http://www.dashif.org 
– Reference Player 
(http://dashif.org/reference/players/javascript) 
• Reference Player Source Code 
– https://github.com/Dash-Industry-Forum/dash.js 
• HTML Extensions 
– MSE: http://www.w3.org/TR/media-source/ 
– EME: http://www.w3.org/TR/encrypted-media/ 
• Twitter 
– @jefftapper 
– @digitalprimates
Questions?

More Related Content

What's hot

H.264 video compression standard.
H.264 video compression standard.H.264 video compression standard.
H.264 video compression standard.Axis Communications
 
2.6 backup and recovery
2.6 backup and recovery2.6 backup and recovery
2.6 backup and recoverymrmwood
 
Bit torrent-technology
Bit torrent-technologyBit torrent-technology
Bit torrent-technologyabhipesit
 
Understanding MPEG DASH
Understanding MPEG DASHUnderstanding MPEG DASH
Understanding MPEG DASHSeung-Bum Lee
 
Introduction to WebRTC
Introduction to WebRTCIntroduction to WebRTC
Introduction to WebRTCArt Matsak
 
Multimedia software tools
Multimedia software toolsMultimedia software tools
Multimedia software toolsJay Patel
 
Removable media up
Removable media upRemovable media up
Removable media upHemanth Raj
 
Web Server And Database Server
Web Server And Database ServerWeb Server And Database Server
Web Server And Database ServerMahbubur Rahman
 
Virtual Machine Migration & Hypervisors
Virtual Machine Migration & HypervisorsVirtual Machine Migration & Hypervisors
Virtual Machine Migration & HypervisorsArun Shukla
 
How to configure dns server(2)
How to configure dns server(2)How to configure dns server(2)
How to configure dns server(2)Amandeep Kaur
 
Chapter 8 : MULTIMEDIA SKILLS
Chapter 8 : MULTIMEDIA SKILLSChapter 8 : MULTIMEDIA SKILLS
Chapter 8 : MULTIMEDIA SKILLSazira96
 
INTRODUCTION TO INTEGRATIVE MEDIA
INTRODUCTION TO INTEGRATIVE MEDIAINTRODUCTION TO INTEGRATIVE MEDIA
INTRODUCTION TO INTEGRATIVE MEDIAAnies Syahieda
 
Streaming Media Protocols
Streaming Media ProtocolsStreaming Media Protocols
Streaming Media Protocolssanjoysanyal
 
Multimedia System & Design Ch 8 delivering
Multimedia System & Design Ch 8 deliveringMultimedia System & Design Ch 8 delivering
Multimedia System & Design Ch 8 deliveringBadar Waseer
 
Streaming Stored Video- Computer Networking
Streaming Stored Video- Computer Networking  Streaming Stored Video- Computer Networking
Streaming Stored Video- Computer Networking Mahbubur Rahman
 

What's hot (20)

H.264 video compression standard.
H.264 video compression standard.H.264 video compression standard.
H.264 video compression standard.
 
2.6 backup and recovery
2.6 backup and recovery2.6 backup and recovery
2.6 backup and recovery
 
Http_Protocol.pptx
Http_Protocol.pptxHttp_Protocol.pptx
Http_Protocol.pptx
 
Bit torrent-technology
Bit torrent-technologyBit torrent-technology
Bit torrent-technology
 
Understanding MPEG DASH
Understanding MPEG DASHUnderstanding MPEG DASH
Understanding MPEG DASH
 
Chapter 14
Chapter 14Chapter 14
Chapter 14
 
Introduction to WebRTC
Introduction to WebRTCIntroduction to WebRTC
Introduction to WebRTC
 
3 multimedia systems
3   multimedia systems3   multimedia systems
3 multimedia systems
 
Multimedia software tools
Multimedia software toolsMultimedia software tools
Multimedia software tools
 
Removable media up
Removable media upRemovable media up
Removable media up
 
Vm migration techniques
Vm migration techniquesVm migration techniques
Vm migration techniques
 
Torrent
TorrentTorrent
Torrent
 
Web Server And Database Server
Web Server And Database ServerWeb Server And Database Server
Web Server And Database Server
 
Virtual Machine Migration & Hypervisors
Virtual Machine Migration & HypervisorsVirtual Machine Migration & Hypervisors
Virtual Machine Migration & Hypervisors
 
How to configure dns server(2)
How to configure dns server(2)How to configure dns server(2)
How to configure dns server(2)
 
Chapter 8 : MULTIMEDIA SKILLS
Chapter 8 : MULTIMEDIA SKILLSChapter 8 : MULTIMEDIA SKILLS
Chapter 8 : MULTIMEDIA SKILLS
 
INTRODUCTION TO INTEGRATIVE MEDIA
INTRODUCTION TO INTEGRATIVE MEDIAINTRODUCTION TO INTEGRATIVE MEDIA
INTRODUCTION TO INTEGRATIVE MEDIA
 
Streaming Media Protocols
Streaming Media ProtocolsStreaming Media Protocols
Streaming Media Protocols
 
Multimedia System & Design Ch 8 delivering
Multimedia System & Design Ch 8 deliveringMultimedia System & Design Ch 8 delivering
Multimedia System & Design Ch 8 delivering
 
Streaming Stored Video- Computer Networking
Streaming Stored Video- Computer Networking  Streaming Stored Video- Computer Networking
Streaming Stored Video- Computer Networking
 

Viewers also liked

MediaPlayer Playing Flow
MediaPlayer Playing FlowMediaPlayer Playing Flow
MediaPlayer Playing FlowJavid Hsu
 
A slightly deeper dive into Stagefright
A slightly deeper dive into StagefrightA slightly deeper dive into Stagefright
A slightly deeper dive into StagefrightAlexy Joseph
 
08 android multimedia_framework_overview
08 android multimedia_framework_overview08 android multimedia_framework_overview
08 android multimedia_framework_overviewArjun Reddy
 
Open Sourcing the Secret Sauce
Open Sourcing the Secret SauceOpen Sourcing the Secret Sauce
Open Sourcing the Secret SauceFITC
 
21st Century Crystal Ball
21st Century Crystal Ball21st Century Crystal Ball
21st Century Crystal BallFITC
 
(Re)aligning The Way 400,000 People Think
(Re)aligning The Way 400,000 People Think(Re)aligning The Way 400,000 People Think
(Re)aligning The Way 400,000 People ThinkFITC
 
PageCloud Reimagines The Future of Website Creation with Craig Fitzpatrick
PageCloud Reimagines The Future of Website Creation with Craig FitzpatrickPageCloud Reimagines The Future of Website Creation with Craig Fitzpatrick
PageCloud Reimagines The Future of Website Creation with Craig FitzpatrickFITC
 
Upgrading the Web with Douglas Crockford @ FITC's Web Unleashed 2015
Upgrading the Web with Douglas Crockford @ FITC's Web Unleashed 2015Upgrading the Web with Douglas Crockford @ FITC's Web Unleashed 2015
Upgrading the Web with Douglas Crockford @ FITC's Web Unleashed 2015FITC
 
Building Tools for the Next Web
Building Tools for the Next WebBuilding Tools for the Next Web
Building Tools for the Next WebFITC
 
Breaking The Broken Web
Breaking The Broken WebBreaking The Broken Web
Breaking The Broken WebFITC
 
The Life of &lt;p>
The Life of &lt;p>The Life of &lt;p>
The Life of &lt;p>FITC
 
A New Era for Animators
A New Era for AnimatorsA New Era for Animators
A New Era for AnimatorsFITC
 
I HATE YOUR GAME with Bob Heubel
I HATE YOUR GAME with Bob HeubelI HATE YOUR GAME with Bob Heubel
I HATE YOUR GAME with Bob HeubelFITC
 
Fitc whats new in es6 for web devs
Fitc   whats new in es6 for web devsFitc   whats new in es6 for web devs
Fitc whats new in es6 for web devsFITC
 
Building Apps with Ember
Building Apps with EmberBuilding Apps with Ember
Building Apps with EmberFITC
 
Ryan Christiani I Heard React Was Good
Ryan Christiani I Heard React Was GoodRyan Christiani I Heard React Was Good
Ryan Christiani I Heard React Was GoodFITC
 
Improving Game Performance in the Browser
Improving Game Performance in the BrowserImproving Game Performance in the Browser
Improving Game Performance in the BrowserFITC
 
Front-end Tools: Sifting Through the Madness
 Front-end Tools: Sifting Through the Madness Front-end Tools: Sifting Through the Madness
Front-end Tools: Sifting Through the MadnessFITC
 
! or ? with Chip Kidd
! or ? with Chip Kidd! or ? with Chip Kidd
! or ? with Chip KiddFITC
 
DownTheRabbitHole.js – How to Stay Sane in an Insane Ecosystem
DownTheRabbitHole.js – How to Stay Sane in an Insane EcosystemDownTheRabbitHole.js – How to Stay Sane in an Insane Ecosystem
DownTheRabbitHole.js – How to Stay Sane in an Insane EcosystemFITC
 

Viewers also liked (20)

MediaPlayer Playing Flow
MediaPlayer Playing FlowMediaPlayer Playing Flow
MediaPlayer Playing Flow
 
A slightly deeper dive into Stagefright
A slightly deeper dive into StagefrightA slightly deeper dive into Stagefright
A slightly deeper dive into Stagefright
 
08 android multimedia_framework_overview
08 android multimedia_framework_overview08 android multimedia_framework_overview
08 android multimedia_framework_overview
 
Open Sourcing the Secret Sauce
Open Sourcing the Secret SauceOpen Sourcing the Secret Sauce
Open Sourcing the Secret Sauce
 
21st Century Crystal Ball
21st Century Crystal Ball21st Century Crystal Ball
21st Century Crystal Ball
 
(Re)aligning The Way 400,000 People Think
(Re)aligning The Way 400,000 People Think(Re)aligning The Way 400,000 People Think
(Re)aligning The Way 400,000 People Think
 
PageCloud Reimagines The Future of Website Creation with Craig Fitzpatrick
PageCloud Reimagines The Future of Website Creation with Craig FitzpatrickPageCloud Reimagines The Future of Website Creation with Craig Fitzpatrick
PageCloud Reimagines The Future of Website Creation with Craig Fitzpatrick
 
Upgrading the Web with Douglas Crockford @ FITC's Web Unleashed 2015
Upgrading the Web with Douglas Crockford @ FITC's Web Unleashed 2015Upgrading the Web with Douglas Crockford @ FITC's Web Unleashed 2015
Upgrading the Web with Douglas Crockford @ FITC's Web Unleashed 2015
 
Building Tools for the Next Web
Building Tools for the Next WebBuilding Tools for the Next Web
Building Tools for the Next Web
 
Breaking The Broken Web
Breaking The Broken WebBreaking The Broken Web
Breaking The Broken Web
 
The Life of &lt;p>
The Life of &lt;p>The Life of &lt;p>
The Life of &lt;p>
 
A New Era for Animators
A New Era for AnimatorsA New Era for Animators
A New Era for Animators
 
I HATE YOUR GAME with Bob Heubel
I HATE YOUR GAME with Bob HeubelI HATE YOUR GAME with Bob Heubel
I HATE YOUR GAME with Bob Heubel
 
Fitc whats new in es6 for web devs
Fitc   whats new in es6 for web devsFitc   whats new in es6 for web devs
Fitc whats new in es6 for web devs
 
Building Apps with Ember
Building Apps with EmberBuilding Apps with Ember
Building Apps with Ember
 
Ryan Christiani I Heard React Was Good
Ryan Christiani I Heard React Was GoodRyan Christiani I Heard React Was Good
Ryan Christiani I Heard React Was Good
 
Improving Game Performance in the Browser
Improving Game Performance in the BrowserImproving Game Performance in the Browser
Improving Game Performance in the Browser
 
Front-end Tools: Sifting Through the Madness
 Front-end Tools: Sifting Through the Madness Front-end Tools: Sifting Through the Madness
Front-end Tools: Sifting Through the Madness
 
! or ? with Chip Kidd
! or ? with Chip Kidd! or ? with Chip Kidd
! or ? with Chip Kidd
 
DownTheRabbitHole.js – How to Stay Sane in an Insane Ecosystem
DownTheRabbitHole.js – How to Stay Sane in an Insane EcosystemDownTheRabbitHole.js – How to Stay Sane in an Insane Ecosystem
DownTheRabbitHole.js – How to Stay Sane in an Insane Ecosystem
 

Similar to Media Source Extensions

Streaming video to html
Streaming video to htmlStreaming video to html
Streaming video to htmljeff tapper
 
Standards' Perspective - MPEG DASH overview and related efforts
Standards' Perspective - MPEG DASH overview and related effortsStandards' Perspective - MPEG DASH overview and related efforts
Standards' Perspective - MPEG DASH overview and related effortsIMTC
 
MPEG DASH White Paper
MPEG DASH White PaperMPEG DASH White Paper
MPEG DASH White Paperidrajeev
 
On-demand & Live Streaming with Amazon CloudFront in the Post-PC World (MED30...
On-demand & Live Streaming with Amazon CloudFront in the Post-PC World (MED30...On-demand & Live Streaming with Amazon CloudFront in the Post-PC World (MED30...
On-demand & Live Streaming with Amazon CloudFront in the Post-PC World (MED30...Amazon Web Services
 
SophiaConf2010 Présentation des Retours d'expériences de la Conférence du 08 ...
SophiaConf2010 Présentation des Retours d'expériences de la Conférence du 08 ...SophiaConf2010 Présentation des Retours d'expériences de la Conférence du 08 ...
SophiaConf2010 Présentation des Retours d'expériences de la Conférence du 08 ...TelecomValley
 
05.m3 cms list-ofwebserver
05.m3 cms list-ofwebserver05.m3 cms list-ofwebserver
05.m3 cms list-ofwebservertarensi
 
haproxy-150423120602-conversion-gate01.pdf
haproxy-150423120602-conversion-gate01.pdfhaproxy-150423120602-conversion-gate01.pdf
haproxy-150423120602-conversion-gate01.pdfPawanVerma628806
 
Silverlight Wireshark Analysis
Silverlight Wireshark AnalysisSilverlight Wireshark Analysis
Silverlight Wireshark AnalysisYoss Cohen
 
ivisa retreat-2019
ivisa retreat-2019ivisa retreat-2019
ivisa retreat-2019chavoosh
 
Windows Azure Media Services June 2013 update
Windows Azure Media Services June 2013 updateWindows Azure Media Services June 2013 update
Windows Azure Media Services June 2013 updateMingfei Yan
 
MPEG-DASH Reference Software and Conformance
MPEG-DASH Reference Software and ConformanceMPEG-DASH Reference Software and Conformance
MPEG-DASH Reference Software and ConformanceAlpen-Adria-Universität
 
System design for video streaming service
System design for video streaming serviceSystem design for video streaming service
System design for video streaming serviceNirmik Kale
 
Media Content Delivery Systems: 2nd Presentation
Media Content Delivery Systems: 2nd PresentationMedia Content Delivery Systems: 2nd Presentation
Media Content Delivery Systems: 2nd Presentationashbyb
 
An Introduction to Open Source Software and Web Application Development
An Introduction to Open Source Software and Web Application DevelopmentAn Introduction to Open Source Software and Web Application Development
An Introduction to Open Source Software and Web Application Developmenttrevorthornton
 
HTML5 Programming
HTML5 ProgrammingHTML5 Programming
HTML5 Programminghotrannam
 
An Evaluation of Dynamic Adaptive Streaming over HTTP in Vehicular Environments
An Evaluation of Dynamic Adaptive Streaming over HTTP in Vehicular EnvironmentsAn Evaluation of Dynamic Adaptive Streaming over HTTP in Vehicular Environments
An Evaluation of Dynamic Adaptive Streaming over HTTP in Vehicular EnvironmentsAlpen-Adria-Universität
 

Similar to Media Source Extensions (20)

Streaming video to html
Streaming video to htmlStreaming video to html
Streaming video to html
 
Standards' Perspective - MPEG DASH overview and related efforts
Standards' Perspective - MPEG DASH overview and related effortsStandards' Perspective - MPEG DASH overview and related efforts
Standards' Perspective - MPEG DASH overview and related efforts
 
MPEG DASH White Paper
MPEG DASH White PaperMPEG DASH White Paper
MPEG DASH White Paper
 
On-demand & Live Streaming with Amazon CloudFront in the Post-PC World (MED30...
On-demand & Live Streaming with Amazon CloudFront in the Post-PC World (MED30...On-demand & Live Streaming with Amazon CloudFront in the Post-PC World (MED30...
On-demand & Live Streaming with Amazon CloudFront in the Post-PC World (MED30...
 
SophiaConf2010 Présentation des Retours d'expériences de la Conférence du 08 ...
SophiaConf2010 Présentation des Retours d'expériences de la Conférence du 08 ...SophiaConf2010 Présentation des Retours d'expériences de la Conférence du 08 ...
SophiaConf2010 Présentation des Retours d'expériences de la Conférence du 08 ...
 
05.m3 cms list-ofwebserver
05.m3 cms list-ofwebserver05.m3 cms list-ofwebserver
05.m3 cms list-ofwebserver
 
haproxy-150423120602-conversion-gate01.pdf
haproxy-150423120602-conversion-gate01.pdfhaproxy-150423120602-conversion-gate01.pdf
haproxy-150423120602-conversion-gate01.pdf
 
HAProxy
HAProxy HAProxy
HAProxy
 
Silverlight Wireshark Analysis
Silverlight Wireshark AnalysisSilverlight Wireshark Analysis
Silverlight Wireshark Analysis
 
ivisa retreat-2019
ivisa retreat-2019ivisa retreat-2019
ivisa retreat-2019
 
Windows Azure Media Services June 2013 update
Windows Azure Media Services June 2013 updateWindows Azure Media Services June 2013 update
Windows Azure Media Services June 2013 update
 
MPEG-DASH Reference Software and Conformance
MPEG-DASH Reference Software and ConformanceMPEG-DASH Reference Software and Conformance
MPEG-DASH Reference Software and Conformance
 
System design for video streaming service
System design for video streaming serviceSystem design for video streaming service
System design for video streaming service
 
Media Content Delivery Systems: 2nd Presentation
Media Content Delivery Systems: 2nd PresentationMedia Content Delivery Systems: 2nd Presentation
Media Content Delivery Systems: 2nd Presentation
 
An Introduction to Open Source Software and Web Application Development
An Introduction to Open Source Software and Web Application DevelopmentAn Introduction to Open Source Software and Web Application Development
An Introduction to Open Source Software and Web Application Development
 
HTML5 Programming
HTML5 ProgrammingHTML5 Programming
HTML5 Programming
 
TechTalk: Connext DDS 5.2.
TechTalk: Connext DDS 5.2.TechTalk: Connext DDS 5.2.
TechTalk: Connext DDS 5.2.
 
Mm sys 2013-demo
Mm sys 2013-demoMm sys 2013-demo
Mm sys 2013-demo
 
An Evaluation of Dynamic Adaptive Streaming over HTTP in Vehicular Environments
An Evaluation of Dynamic Adaptive Streaming over HTTP in Vehicular EnvironmentsAn Evaluation of Dynamic Adaptive Streaming over HTTP in Vehicular Environments
An Evaluation of Dynamic Adaptive Streaming over HTTP in Vehicular Environments
 
Multimedia Streaming Architecture
Multimedia Streaming ArchitectureMultimedia Streaming Architecture
Multimedia Streaming Architecture
 

More from FITC

Cut it up
Cut it upCut it up
Cut it upFITC
 
Designing for Digital Health
Designing for Digital HealthDesigning for Digital Health
Designing for Digital HealthFITC
 
Profiling JavaScript Performance
Profiling JavaScript PerformanceProfiling JavaScript Performance
Profiling JavaScript PerformanceFITC
 
Surviving Your Tech Stack
Surviving Your Tech StackSurviving Your Tech Stack
Surviving Your Tech StackFITC
 
How to Pitch Your First AR Project
How to Pitch Your First AR ProjectHow to Pitch Your First AR Project
How to Pitch Your First AR ProjectFITC
 
Start by Understanding the Problem, Not by Delivering the Answer
Start by Understanding the Problem, Not by Delivering the AnswerStart by Understanding the Problem, Not by Delivering the Answer
Start by Understanding the Problem, Not by Delivering the AnswerFITC
 
Cocaine to Carrots: The Art of Telling Someone Else’s Story
Cocaine to Carrots: The Art of Telling Someone Else’s StoryCocaine to Carrots: The Art of Telling Someone Else’s Story
Cocaine to Carrots: The Art of Telling Someone Else’s StoryFITC
 
Everyday Innovation
Everyday InnovationEveryday Innovation
Everyday InnovationFITC
 
HyperLight Websites
HyperLight WebsitesHyperLight Websites
HyperLight WebsitesFITC
 
Everything is Terrifying
Everything is TerrifyingEverything is Terrifying
Everything is TerrifyingFITC
 
Post-Earth Visions: Designing for Space and the Future Human
Post-Earth Visions: Designing for Space and the Future HumanPost-Earth Visions: Designing for Space and the Future Human
Post-Earth Visions: Designing for Space and the Future HumanFITC
 
The Rise of the Creative Social Influencer (and How to Become One)
The Rise of the Creative Social Influencer (and How to Become One)The Rise of the Creative Social Influencer (and How to Become One)
The Rise of the Creative Social Influencer (and How to Become One)FITC
 
East of the Rockies: Developing an AR Game
East of the Rockies: Developing an AR GameEast of the Rockies: Developing an AR Game
East of the Rockies: Developing an AR GameFITC
 
Creating a Proactive Healthcare System
Creating a Proactive Healthcare SystemCreating a Proactive Healthcare System
Creating a Proactive Healthcare SystemFITC
 
World Transformation: The Secret Agenda of Product Design
World Transformation: The Secret Agenda of Product DesignWorld Transformation: The Secret Agenda of Product Design
World Transformation: The Secret Agenda of Product DesignFITC
 
The Power of Now
The Power of NowThe Power of Now
The Power of NowFITC
 
High Performance PWAs
High Performance PWAsHigh Performance PWAs
High Performance PWAsFITC
 
Rise of the JAMstack
Rise of the JAMstackRise of the JAMstack
Rise of the JAMstackFITC
 
From Closed to Open: A Journey of Self Discovery
From Closed to Open: A Journey of Self DiscoveryFrom Closed to Open: A Journey of Self Discovery
From Closed to Open: A Journey of Self DiscoveryFITC
 
Projects Ain’t Nobody Got Time For
Projects Ain’t Nobody Got Time ForProjects Ain’t Nobody Got Time For
Projects Ain’t Nobody Got Time ForFITC
 

More from FITC (20)

Cut it up
Cut it upCut it up
Cut it up
 
Designing for Digital Health
Designing for Digital HealthDesigning for Digital Health
Designing for Digital Health
 
Profiling JavaScript Performance
Profiling JavaScript PerformanceProfiling JavaScript Performance
Profiling JavaScript Performance
 
Surviving Your Tech Stack
Surviving Your Tech StackSurviving Your Tech Stack
Surviving Your Tech Stack
 
How to Pitch Your First AR Project
How to Pitch Your First AR ProjectHow to Pitch Your First AR Project
How to Pitch Your First AR Project
 
Start by Understanding the Problem, Not by Delivering the Answer
Start by Understanding the Problem, Not by Delivering the AnswerStart by Understanding the Problem, Not by Delivering the Answer
Start by Understanding the Problem, Not by Delivering the Answer
 
Cocaine to Carrots: The Art of Telling Someone Else’s Story
Cocaine to Carrots: The Art of Telling Someone Else’s StoryCocaine to Carrots: The Art of Telling Someone Else’s Story
Cocaine to Carrots: The Art of Telling Someone Else’s Story
 
Everyday Innovation
Everyday InnovationEveryday Innovation
Everyday Innovation
 
HyperLight Websites
HyperLight WebsitesHyperLight Websites
HyperLight Websites
 
Everything is Terrifying
Everything is TerrifyingEverything is Terrifying
Everything is Terrifying
 
Post-Earth Visions: Designing for Space and the Future Human
Post-Earth Visions: Designing for Space and the Future HumanPost-Earth Visions: Designing for Space and the Future Human
Post-Earth Visions: Designing for Space and the Future Human
 
The Rise of the Creative Social Influencer (and How to Become One)
The Rise of the Creative Social Influencer (and How to Become One)The Rise of the Creative Social Influencer (and How to Become One)
The Rise of the Creative Social Influencer (and How to Become One)
 
East of the Rockies: Developing an AR Game
East of the Rockies: Developing an AR GameEast of the Rockies: Developing an AR Game
East of the Rockies: Developing an AR Game
 
Creating a Proactive Healthcare System
Creating a Proactive Healthcare SystemCreating a Proactive Healthcare System
Creating a Proactive Healthcare System
 
World Transformation: The Secret Agenda of Product Design
World Transformation: The Secret Agenda of Product DesignWorld Transformation: The Secret Agenda of Product Design
World Transformation: The Secret Agenda of Product Design
 
The Power of Now
The Power of NowThe Power of Now
The Power of Now
 
High Performance PWAs
High Performance PWAsHigh Performance PWAs
High Performance PWAs
 
Rise of the JAMstack
Rise of the JAMstackRise of the JAMstack
Rise of the JAMstack
 
From Closed to Open: A Journey of Self Discovery
From Closed to Open: A Journey of Self DiscoveryFrom Closed to Open: A Journey of Self Discovery
From Closed to Open: A Journey of Self Discovery
 
Projects Ain’t Nobody Got Time For
Projects Ain’t Nobody Got Time ForProjects Ain’t Nobody Got Time For
Projects Ain’t Nobody Got Time For
 

Recently uploaded

Google-Next-Madrid-BBVA-Research inv.pdf
Google-Next-Madrid-BBVA-Research inv.pdfGoogle-Next-Madrid-BBVA-Research inv.pdf
Google-Next-Madrid-BBVA-Research inv.pdfMaria Adalfio
 
Section 3 - Technical Sales Foundations for IBM QRadar for Cloud (QRoC)V1 P10...
Section 3 - Technical Sales Foundations for IBM QRadar for Cloud (QRoC)V1 P10...Section 3 - Technical Sales Foundations for IBM QRadar for Cloud (QRoC)V1 P10...
Section 3 - Technical Sales Foundations for IBM QRadar for Cloud (QRoC)V1 P10...hasimatwork
 
Generalities about NFT , as a new technology
Generalities about NFT , as a new technologyGeneralities about NFT , as a new technology
Generalities about NFT , as a new technologysoufianbouktaib1
 
如何办理朴茨茅斯大学毕业证书学位证书成绩单?
如何办理朴茨茅斯大学毕业证书学位证书成绩单?如何办理朴茨茅斯大学毕业证书学位证书成绩单?
如何办理朴茨茅斯大学毕业证书学位证书成绩单?krc0yvm5
 
Mary Meeker Internet Trends Report for 2019
Mary Meeker Internet Trends Report for 2019Mary Meeker Internet Trends Report for 2019
Mary Meeker Internet Trends Report for 2019Eric Johnson
 
SQL Server on Azure VM datasheet.dsadaspptx
SQL Server on Azure VM datasheet.dsadaspptxSQL Server on Azure VM datasheet.dsadaspptx
SQL Server on Azure VM datasheet.dsadaspptxJustineGarcia32
 
APNIC Update and RIR Policies for ccTLDs, presented at APTLD 85
APNIC Update and RIR Policies for ccTLDs, presented at APTLD 85APNIC Update and RIR Policies for ccTLDs, presented at APTLD 85
APNIC Update and RIR Policies for ccTLDs, presented at APTLD 85APNIC
 
Tungsten Webinar: v6 & v7 Release Recap, and Beyond
Tungsten Webinar: v6 & v7 Release Recap, and BeyondTungsten Webinar: v6 & v7 Release Recap, and Beyond
Tungsten Webinar: v6 & v7 Release Recap, and BeyondContinuent
 
Benefits of Fiber Internet vs. Traditional Internet.pptx
Benefits of Fiber Internet vs. Traditional Internet.pptxBenefits of Fiber Internet vs. Traditional Internet.pptx
Benefits of Fiber Internet vs. Traditional Internet.pptxlibertyuae uae
 
overview of Virtualization, concept of Virtualization
overview of Virtualization, concept of Virtualizationoverview of Virtualization, concept of Virtualization
overview of Virtualization, concept of VirtualizationRajan yadav
 

Recently uploaded (10)

Google-Next-Madrid-BBVA-Research inv.pdf
Google-Next-Madrid-BBVA-Research inv.pdfGoogle-Next-Madrid-BBVA-Research inv.pdf
Google-Next-Madrid-BBVA-Research inv.pdf
 
Section 3 - Technical Sales Foundations for IBM QRadar for Cloud (QRoC)V1 P10...
Section 3 - Technical Sales Foundations for IBM QRadar for Cloud (QRoC)V1 P10...Section 3 - Technical Sales Foundations for IBM QRadar for Cloud (QRoC)V1 P10...
Section 3 - Technical Sales Foundations for IBM QRadar for Cloud (QRoC)V1 P10...
 
Generalities about NFT , as a new technology
Generalities about NFT , as a new technologyGeneralities about NFT , as a new technology
Generalities about NFT , as a new technology
 
如何办理朴茨茅斯大学毕业证书学位证书成绩单?
如何办理朴茨茅斯大学毕业证书学位证书成绩单?如何办理朴茨茅斯大学毕业证书学位证书成绩单?
如何办理朴茨茅斯大学毕业证书学位证书成绩单?
 
Mary Meeker Internet Trends Report for 2019
Mary Meeker Internet Trends Report for 2019Mary Meeker Internet Trends Report for 2019
Mary Meeker Internet Trends Report for 2019
 
SQL Server on Azure VM datasheet.dsadaspptx
SQL Server on Azure VM datasheet.dsadaspptxSQL Server on Azure VM datasheet.dsadaspptx
SQL Server on Azure VM datasheet.dsadaspptx
 
APNIC Update and RIR Policies for ccTLDs, presented at APTLD 85
APNIC Update and RIR Policies for ccTLDs, presented at APTLD 85APNIC Update and RIR Policies for ccTLDs, presented at APTLD 85
APNIC Update and RIR Policies for ccTLDs, presented at APTLD 85
 
Tungsten Webinar: v6 & v7 Release Recap, and Beyond
Tungsten Webinar: v6 & v7 Release Recap, and BeyondTungsten Webinar: v6 & v7 Release Recap, and Beyond
Tungsten Webinar: v6 & v7 Release Recap, and Beyond
 
Benefits of Fiber Internet vs. Traditional Internet.pptx
Benefits of Fiber Internet vs. Traditional Internet.pptxBenefits of Fiber Internet vs. Traditional Internet.pptx
Benefits of Fiber Internet vs. Traditional Internet.pptx
 
overview of Virtualization, concept of Virtualization
overview of Virtualization, concept of Virtualizationoverview of Virtualization, concept of Virtualization
overview of Virtualization, concept of Virtualization
 

Media Source Extensions

  • 1. Media Source Extensions Streaming Video without Plugins Jeff Tapper Digital Primates @jefftapper
  • 2. Who am I? • Senior Consultant at Digital Primates – Building next generation client applications • Built video applications for many of the most watched live broadcasts • Developing Internet applications for 19 years • Author of 12 books on Internet technologies
  • 4. Agenda • Video and the Internet today • Understanding HTTP Streaming • What are the Streaming options without a plugin? • What is DASH • What is DASH-264 • Making it work in a browser • Questions
  • 5. Online video Options • Progressive Download • Real Time Protocols (RTP, RTMP, RTSP, etc) • HTTP Streaming (HDS, HLS, Smooth Streaming, etc)
  • 6. The challenge • Most agree that HTTP Streaming is the most efficient choice • Different devices support different streaming protocols • No one standard is currently supported ubiquitously • Results in media being served in several different formats to support the broadest range of devices
  • 7. What do browsers support? • Unfortunately, Progressive Download is the only ubiquitously supported option • Different Browsers support different video codec’s – H.264 – webM – VP8/VP9 – Etc. • Safari (iOs and MacOS only) natively supports HLS • MediaSource Extensions released in Chrome and IE11, betas in Safari and Firefox
  • 8. MediaSource Extensions (MSE) • MSE allow for pieces (segments) of media to be handed to the HTML5 video tag’s buffer directly. • This enables HTTP Streaming in HTML • Not universally supported, yet. • Currently (as of September 2014) an Editors Draft to the HTML Working Group
  • 9. What is MPEG-DASH  DASH – Dynamic Adaptive Streaming via HTTP  International open standard, developed and published by ISO  Addresses both simple and advanced use cases  Enables highest-quality multiscreen distribution and efficient dynamic adaptive switching  Enables reuse of existing content, devices and infrastructure  Attempts to unify to a single standard for HTTP Streaming
  • 10. DASH and codecs • The DASH specification is codec agnostic • Any existing or future codec can work with DASH • DASH manifest describes which codec is used • Allows ability for a single manifest to describe several different versions in different codecs
  • 11. DASH264 • H.264 is dominant format today • Many vendors and service providers are committed to supporting/enabling DASH264 • Provides support for today’s requirements such as DRM • H.264 is backed by rigorous testing and conformance
  • 12. DASH Industry Forum • Addressing the dramatic growth of broadband video by recommending a universal delivery format that provides end users with the best possible media experience by dynamically adapting to changing network conditions.
  • 13. DASH Industry Forum • Objectives: – promote and catalyze market adoption of MPEG-DASH – publish interoperability and deployment guidelines – facilitate interoperability tests – collaborate with standard bodies and industry consortia in aligning ongoing DASH standards development and the use of common profiles across industry organizations • Over 65 members • Visit http://dashif.org for more information • Released the DASH/264 standard
  • 14. Building a DASH player • We have built DASH players for several different platforms – Flash – Android – HTML5/JavaScript (dash.js) • DASH.js is available as an open source project (bsd3) on github • DASH.js is the reference player for the DASH Industry Forum (dashif.org)
  • 15. How to play a DASH Stream • Download Manifest • Parse Manifest • Determine optimal bandwidth for client • Initialize for bandwidth • Download Segment • Hand segment to MSE • Check Bandwidth to determine if change is necessary
  • 16. Understanding DASH structure • Three types of files – Manifest (.mpd) • XML file describing the segments – Initialization file • Contains headers needed to decode bytes in segments – Segment Files • Contains playable media • Includes: – 0…many video tracks – 0…many audio tracks
  • 17. DASH Manifest • Manifest contains: – Root node – 1 or more periods • Periods contain 1 adaptation set per video stream and • Periods contain 1 adaptation set per audio stream • Adaptation Sets contain: – Content Composition nodes (for each video or audio track) – 1 or more Representation node » Each representation describes a single bitrate » Representations contain data on finding the actual segments » Different ways a representation can describe segments
  • 18. Describing Representations • SegmentBase – Describes a stream with only a single Segment per bitrate – Can be used for Byte Range Requests • SegmentList – A SegmentList will contain a specific list of each SegmentURL (individual HTTP packet with media data) – Can be used for Byte Range Requests • SegmentTemplate – Defines a known url for the fragment with wildcards resolved at runtime to request a segments (see bbb.mpd) – Alternatively, can specify a list of segments based on duration
  • 19. SegmentList <Representation id="h264bl_hd" mimeType="video/mp4" codecs="avc1.42c01f" width="1280" height="720" startWithSAP="1" bandwidth="514864"> <SegmentList timescale="1000" duration="10000"> <Initialization sourceURL="mp4-main-multi-h264bl_hd-.mp4"/> <SegmentURL media="mp4-main-multi-h264bl_hd-1.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-2.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-3.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-4.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-5.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-6.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-7.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-8.m4s"/>
  • 20. SegmentTemplate fixed segment duration <AdaptationSet> <ContentComponent id="1" contentType="video"/> <SegmentTemplate initialization="BigBuckBunny_720p_1800kbps_44khz_track1_dash.mp4"/> <Representation id="1" mimeType="video/mp4“ codecs="avc1.64001f" width="1280" height="720“ startWithSAP="1" bandwidth="1809954"> <SegmentTemplate timescale="1000" duration="13809" media="bbb_seg_BigBuckBunny_720p_1800kbps_44khz_track1$Number$.m4s" startNumber="1"/> </Representation> </AdaptationSet>
  • 21. SegmentTemplate variable segment duration <AdaptationSet group="2" mimeType="video/mp4" par="16:9“ minBandwidth="475000“ maxBandwidth="6589000" minWidth="176" maxWidth="1680" minHeight="99" maxHeight="944“ segmentAlignment="true“ startWithSAP="1"> <SegmentTemplate timescale="1000" initialization="dash/ateam-video=$Bandwidth$.dash" media="dash/ateam-video=$Bandwidth$-$Time$.dash"> <SegmentTimeline> <S t="0" d="4171" /> <S d="2503" /> <S d="2961" /> <S d="2461" /> <S d="2127" r="2" /> …
  • 23. dash.js player • dash.js is a free open source player • Code available on github • Currently the base of several different production players • Recent uses include: – BBC live broadcasts – Wowza – EZDRM – And more!
  • 24. How to play a DASH Stream • Download Manifest • Parse Manifest • Determine optimal bandwidth for client • Initialize for bandwidth • Download Segment • Hand segment to MSE • Check Bandwidth to determine if change is necessary
  • 25. Tools used by dash.js Core Player • Dijon – DI / IOC Development • Jasmine – unit tests Web Site • AngularJS – Application Framework • Flat-ui – UI elements • Flot – Charting • Kendo - Components
  • 26. Class Structure • The player is divided into two main packages. • streaming – Contains the classes responsible for creating and populating the MediaSource buffers. These classes are intended to be abstract enough for use with any segmented stream (such as DASH, HLS, HDS and MSS). • dash – Contains the classes responsible for making decisions specifically related to Dash.
  • 28. MediaPlayer.js • Exposes the top level functions and properties to the developer (play, autoPlay, isLive, abr quality, and metrics). • The manifest URL and the HTML Video object as passed to the MediaPlayer.
  • 29. Context.js • The dependency mapping for the stream package. • The context is passed into the MediaPlayer object allowing for different MediaPlayer instances to use different mappings.
  • 30. Stream.js • Loads/refreshes the manifest. • Create SourceBuffers from MediaSource. • Create BufferManager classes to manage SourceBuffers. • Responds to events from HTML Video object. • For a live stream, the live edge is calculated and passed to the BufferController instances.
  • 31. Debug.js • Convenience class for logging methods. • Default implementation is to just use console.log(). • Extension point for tapping into logging messages.
  • 32. BufferController.js • Responsible for loading fragments and pushing the bytes into the SourceBuffer. • Once play() has been called a timer is started to check the status of the bytes in the buffer. • If the amount of time left to play is less than Manifest.minBufferTime the next fragment is loaded. • Records metrics related to playback.
  • 33. ManifestLoader.js • Responsible for loading manifest files. • Returns the parsed manifest object. FragmentLoader.js • Responsible for loading fragments. • Loads requests sequentially.
  • 34. AbrController.js • Responsible for deciding if the current quality should be changed. • The stream metrics are passed to a set of ‘rules’. • Methods: getPlaybackQuality(type, data)  type – The type of the data (audio/video).  data – The stream data.
  • 35. DownloadRatioRule.js • Validates that fragments are being downloaded in a timely manner. • Compares the time it takes to download a fragment to how long it takes to play out a fragment. • If the download time is considered a bottleneck the quality will be lowered.
  • 36. InsufficientBufferRule.js • Validates that the buffer doesn’t run dry during playback. • If the buffer is running dry continuously it likely means that the player has a processing bottleneck (video decode time is longer than playback time).
  • 38. DashContext.js • Defines dependency mapping specific to the dash package. – Parser – Index Handler – Manifest Extensions
  • 39. DashParser.js • Converts the manifest to a JSON object. • Converts duration and datetime strings into number/date objects. • Manages inheritance fields. – Many fields are inherited from parent to child nodes in DASH. – For example, a BaseURL can be defined in the <MPD> node and all <Representation> nodes inherit that value.
  • 40. DashHandler.js • Responsible for deciding which fragment URL should be loaded. • Methods:  getInitRequest(quality) – Returns an initialization request for a given quality, if available.  getSegmentRequestForTime(time, quality) – Returns a fragment URL to load for a given quality and a given time. Returns a Stream.vo.SegmentRequest object.  getNextSegmentRequest(quality) – Returns the next fragment URL to load. Assumes that getSegmentRequestForTime() has already been called.  getCurrentTime (quality) – Returns the time for the last loaded fragment index.
  • 41. DashHandler.js (cont’d) • Uses available information in the manifest (SegmentList, SegmentTemplate, SegmentBase). • When using a single, non-fragmented mp4 file the SIDX box will be loaded to determine byte ranges for segments.
  • 42. Flow 1. Create the Context and MediaPlayer instances. var context = new Dash.di.DashContext(), player = new MediaPlayer(context); 2. Initialize MediaPlayer and set manifest URL. player.startup(); player.setIsLive(false); player.attachSource(manifest_url); 3. Attach HTML Video element. video = document.querySelector(".dash-video-player video"), player.autoPlay = true; player.attachView(video);
  • 43. 2. Call play()on the MediaPlayer (if autoPlay = false). 3. The Stream object will be created and initialized with the manifest URL. 4. The manifest is loaded and then parsed. 5.MediaSource, SourceBuffers, and BufferControllers are created. – Create one BufferController per stream type (usually video and audio). 6. Set the duration of the MediaSource to the duration of the manifest (or infinity for a live stream). 7. If the stream is live, calculate the live edge. 8. Call play() on the HTML video element. 9. The BufferManager instances create a timer. When the timer ticks the state of the buffers is checked.
  • 44. BufferManager.validate() 1. Check to see if the buffers need more data. • Must be in a playing state. • Must not already be loading data. • Must require more data to be buffered. amountBuffered < manifest.minBufferTime 2. If automatic ABR is enabled check to see if the bitrate should be changed. • Ask AbrController for the new quality. • Rules will determine which bitrate to change to. 3. If initial playback, seeking, or the bitrate has changed load the initialization fragment (if available).
  • 45. 4. Ask the IndexHandler for the next fragment request. • If seeking pass the seek time to the IndexHandler. • Otherwise ask for the ‘next’ fragment. • Pass the bitrate to the IndexHandler. 6. The IndexHandler returns a SegmentRequest indicating what action the BufferManager should take next. • “download” – Download and append the fragment to the buffer. • “stall” – Wait because the IndexHandler is not ready. • “complete” – Signal that the stream has completed playback. 7. Repeat.
  • 46. Resources • DASH Industry Forum – http://www.dashif.org – Reference Player (http://dashif.org/reference/players/javascript) • Reference Player Source Code – https://github.com/Dash-Industry-Forum/dash.js • HTML Extensions – MSE: http://www.w3.org/TR/media-source/ – EME: http://www.w3.org/TR/encrypted-media/ • Twitter – @jefftapper – @digitalprimates