Building a Dash-264 Player    Jeff Tapper / Michael Labriola            Digital Primates     jtapper@digitalprimates.net  ...
Jeff Tapper• Senior Consultant at Digital Primates  – Building next generation client applications• Built video applicatio...
Who am I?• Michael Labriola• Senior Consultant at Digital Primates  – Building next generation client applications• Built ...
Agenda•   Video and the Internet today•   Understanding HTTP Streaming•   What is DASH•   Flash as a video platform•   The...
Video is dominating the Internet• Desktop: Video makes up 50% of traffic at peak periods  –   notably 30% from Netflix and...
Encode each segment                                      HTTP Adaptive Streaming  1           at multiple bitrates        ...
HTTP Streaming Landscape•   Apple’s HTTP Live Streaming (HLS)•   Microsoft’s Smooth Streaming•   Adobe’s HTTP Dynamic Stre...
The challenge• Most agree that HTTP Streaming is the most  efficient choice• Different devices support different streaming...
What is MPEG-DASH DASH – Dynamic Adaptive Streaming via HTTP International open standard, developed and  published by IS...
DASH and codecs• The DASH specification is codec agnostic• Any existing or future codec can work with  DASH• DASH manifest...
DASH264• H.264 is dominant format today• Many vendors and service providers are  committed to supporting/enabling DASH264•...
DASH Industry Forum• Addressing the dramatic growth of broadband  video by recommending a universal delivery  format that ...
DASH Industry Forum• Objectives:   –   promote and catalyze market adoption of MPEG-DASH   –   publish interoperability an...
Members
Building a Desktop DASH player• There are many technologies available for  playing video in modern web browsers  – Flash  ...
Building Video Players in Flash• Flash Player includes a native video player  – flash.media.Video• Video accepts a source ...
Flash Video (continued)• NetStream and Video classes can be  manipulated manually• Open Source frameworks are available fo...
OSMF• Extensible framework with plugin architecture• Native support for DRM• Contains classes to work with HTTP Streaming ...
How to DASH in Flash• Extract Video Bytes from DASH segments• Wrap bytes in format understood by flash  (FLV)• Append FLV ...
Understanding DASH structure• Three types of files  – Manifest (.mpd)     • XML file describing the segments  – Initializa...
DASH Manifest• Manifest contains:  – Program Info (including title)  – 1 or more periods     • Periods contain 1 adaptatio...
Describing Representations• SegmentBase  – Describes a stream with only a single Segment per bitrate  – Can be used for By...
SegmentList<Representation id="h264bl_hd" mimeType="video/mp4"codecs="avc1.42c01f" width="1280" height="720" startWithSAP=...
SegmentTemplate fixed segment                duration<AdaptationSet>  <ContentComponent id="1" contentType="video"/>  <Seg...
SegmentTemplate variable segment             duration<AdaptationSet group="2" mimeType="video/mp4" par="16:9“ minBandwidth...
Understanding MP4 – H.264 structure• Each segment is binary, with a series of boxes  contained within• First 8 bytes of ev...
Initialization Segment• FTYP – file type and compatibility• FREE – free space• MOOV – container of all metadata  – MVHD – ...
Content Segments• STYP - segment type• SIDX - segment index• MOOF -movie fragment  – MFHD - movie fragment header  – TRAF ...
Pulling it all together• Using OSMF, we wrote a DASH specific  IndexHandler and FileHandler• IndexHandler responsible for ...
DashIndexHandler• When OSMF determines a new fragment  needs to be loaded, it calls the  DashIndexHandler.• When playing, ...
DashFileHandler• When the requested file is returned, it is handed  to the DashFileHandler for processing  – initializePro...
MP4Decoder• processData() – iterates through all boxes in  segment  – Calls getNextBox()• getNextBox knows how to process ...
Digital Primates DASH Player• Currently our DASH player is not commercially  available• We use it as the basis for DASH pl...
Current DP Dash Player• We currently support SegmentLists,  SegmentTemplates and SegmentBase in manifest  – Byte Range Req...
DP Dash Player plans• 4Q12                       • 1Q13  – Basic Seek Fixes            – Initial JavaScript Player  – Impr...
Resources• GPAC  – http://gpac.wp.mines-telecom.fr  – Provides baseline test streams  – Provides baseline player• MP4Parse...
Questions?
Upcoming SlideShare
Loading in...5
×

Building a Dash-264 Player

12,246

Published on

Slides from Streaming Media West Presentation, "How to Create A DASH-264 player for Adobe Flash"

Published in: Technology
0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
12,246
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
127
Comments
0
Likes
10
Embeds 0
No embeds

No notes for slide

Building a Dash-264 Player

  1. 1. Building a Dash-264 Player Jeff Tapper / Michael Labriola Digital Primates jtapper@digitalprimates.net mlabriola@digitalprimates.net Twitter: @jtapper / @mlabriola
  2. 2. Jeff Tapper• Senior Consultant at Digital Primates – Building next generation client applications• Built video applications for clients including …• Developing Internet applications for 17 years• Author of 12 books on Internet technologies
  3. 3. Who am I?• Michael Labriola• Senior Consultant at Digital Primates – Building next generation client applications• Built video applications for clients including …well lots…• Developing Internet applications for 17 years• Author of 5 books on Internet technologies
  4. 4. Agenda• Video and the Internet today• Understanding HTTP Streaming• What is DASH• Flash as a video platform• The inner workings of DASH• The challenge• Getting video bytes info flash• Introducing the Digital Primates DASH-264 Player• Questions
  5. 5. Video is dominating the Internet• Desktop: Video makes up 50% of traffic at peak periods – notably 30% from Netflix and 11% from YouTube• Mobile: Video traffic is growing exponentially Fixed Internet Mobile Internet
  6. 6. Encode each segment HTTP Adaptive Streaming 1 at multiple bitrates Split the video into 2 small segments Client splices together 5 and plays back Make each segment 0010101000 01010 0101010100 01110 0111010001 1010101 3 addressable via a HTTP-URL 0010101000010 10 0101010100011 10 0111010001101 0101 0010101000 0101010100 0111010001 01010 01110 1010101 0010101000 0101010100 0111010001 01010 01110 1010101 Client makes decisionMedia Capture & Encoding 4 on which segment to download Media Origin Servers HTTP Cache Servers Client Devices
  7. 7. HTTP Streaming Landscape• Apple’s HTTP Live Streaming (HLS)• Microsoft’s Smooth Streaming• Adobe’s HTTP Dynamic Streaming (HDS)• And many more…
  8. 8. The challenge• Most agree that HTTP Streaming is the most efficient choice• Different devices support different streaming protocols• No one standard is currently supported ubiquitously• Results in media being served in several different formats to support the broadest range of devices
  9. 9. What is MPEG-DASH DASH – Dynamic Adaptive Streaming via HTTP International open standard, developed and published by ISO Addresses both simple and advanced use cases Enables highest-quality multiscreen distribution and efficient dynamic adaptive switching Enables reuse of existing content, devices and infrastructure Attempts to unify to a single standard for HTTP Streaming
  10. 10. DASH and codecs• The DASH specification is codec agnostic• Any existing or future codec can work with DASH• DASH manifest describes which codec is used – Different codecs store the actual video data differently
  11. 11. DASH264• H.264 is dominant format today• Many vendors and service providers are committed to supporting/enabling DASH264• Provides support for today’s requirements such as DRM• H.264 is backed by rigorous testing and conformance
  12. 12. DASH Industry Forum• Addressing the dramatic growth of broadband video by recommending a universal delivery format that provides end users with the best possible media experience by dynamically adapting to changing network conditions.
  13. 13. DASH Industry Forum• Objectives: – promote and catalyze market adoption of MPEG-DASH – publish interoperability and deployment guidelines – facilitate interoperability tests – collaborate with standard bodies and industry consortia in aligning ongoing DASH standards development and the use of common profiles across industry organizations• Currently 46 members• Visit http://dashif.org for more information
  14. 14. Members
  15. 15. Building a Desktop DASH player• There are many technologies available for playing video in modern web browsers – Flash – SilverLight – HTML5 – Etc.• We focused on Flash for our first DASH player – Largest percentage of users can support it
  16. 16. Building Video Players in Flash• Flash Player includes a native video player – flash.media.Video• Video accepts a source from a Camera or a NetStream – flash.media.Camera – flash.net.NetStream• Netstream supports: – Streaming FLV file – Manually appending FLV Bytes into its buffer
  17. 17. Flash Video (continued)• NetStream and Video classes can be manipulated manually• Open Source frameworks are available for easing the process – Adobe’s Open Source Media Framework (OSMF) – Open Video Player (OVP)• We have chosen to build on OSMF
  18. 18. OSMF• Extensible framework with plugin architecture• Native support for DRM• Contains classes to work with HTTP Streaming technologies – Only HDS is supported natively – Extensible architecture makes it possible to support new technologies
  19. 19. How to DASH in Flash• Extract Video Bytes from DASH segments• Wrap bytes in format understood by flash (FLV)• Append FLV Bytes into Netstream
  20. 20. Understanding DASH structure• Three types of files – Manifest (.mpd) • XML file describing the segments – Initialization file • Contains headers needed to decode bytes in segments – Segment Files • Contains playable media • Includes: – 0…many video tracks – 0…many audio tracks
  21. 21. DASH Manifest• Manifest contains: – Program Info (including title) – 1 or more periods • Periods contain 1 adaptation set per video stream and • Periods contain 1 adaptation set per audio stream • Adaptation Sets contain: – Content Composition nodes (for each video or audio track) – 1 or more Representation node » Each representation describes a single bitrate » Representations contain data on finding the actual segments » Different ways a representation can describe segments
  22. 22. Describing Representations• SegmentBase – Describes a stream with only a single Segment per bitrate – Can be used for Byte Range Requests• SegmentList – A SegmentList will contain a specific list of each SegmentURL (individual HTTP packet with media data) – Can be used for Byte Range Requests• SegmentTemplate – Defines a known url for the fragment with wildcards resolved at runtime to request a segments (see bbb.mpd) – Alternatively, can specify a list of segments based on duration
  23. 23. SegmentList<Representation id="h264bl_hd" mimeType="video/mp4"codecs="avc1.42c01f" width="1280" height="720" startWithSAP="1"bandwidth="514864"> <SegmentList timescale="1000" duration="10000"> <Initialization sourceURL="mp4-main-multi-h264bl_hd-.mp4"/> <SegmentURL media="mp4-main-multi-h264bl_hd-1.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-2.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-3.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-4.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-5.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-6.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-7.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-8.m4s"/>
  24. 24. SegmentTemplate fixed segment duration<AdaptationSet> <ContentComponent id="1" contentType="video"/> <SegmentTemplateinitialization="BigBuckBunny_720p_1800kbps_44khz_track1_dash.mp4"/> <Representation id="1" mimeType="video/mp4“ codecs="avc1.64001f" width="1280" height="720“ startWithSAP="1" bandwidth="1809954"> <SegmentTemplate timescale="1000" duration="13809" media="bbb_seg_BigBuckBunny_720p_1800kbps_44khz_track1$Number$.m4s" startNumber="1"/> </Representation></AdaptationSet>
  25. 25. SegmentTemplate variable segment duration<AdaptationSet group="2" mimeType="video/mp4" par="16:9“ minBandwidth="475000“ maxBandwidth="6589000" minWidth="176" maxWidth="1680" minHeight="99" maxHeight="944“ segmentAlignment="true“ startWithSAP="1"> <SegmentTemplate timescale="1000" initialization="dash/ateam-video=$Bandwidth$.dash" media="dash/ateam-video=$Bandwidth$-$Time$.dash"> <SegmentTimeline> <S t="0" d="4171" /> <S d="2503" /> <S d="2961" /> <S d="2461" /> <S d="2127" r="2" />…
  26. 26. Understanding MP4 – H.264 structure• Each segment is binary, with a series of boxes contained within• First 8 bytes of every box defines – Box type – Number of bytes for the box.
  27. 27. Initialization Segment• FTYP – file type and compatibility• FREE – free space• MOOV – container of all metadata – MVHD – movie header – 1.* TRAK – container for individual stream data • TKHD – track header • MDIA – container for all media info – Lots of child boxes which contain more detailed stream info
  28. 28. Content Segments• STYP - segment type• SIDX - segment index• MOOF -movie fragment – MFHD - movie fragment header – TRAF - track fragment • TFHD - track fragment header • TFDT - track fragment decode time • TRUN - track fragment run• MDAT - media data container (actual media data) – Contains one or more samples (smallest described piece of content)
  29. 29. Pulling it all together• Using OSMF, we wrote a DASH specific IndexHandler and FileHandler• IndexHandler responsible for deciding url of next http request• FileHandler responsible for parsing data from segments and returning FLV Bytes
  30. 30. DashIndexHandler• When OSMF determines a new fragment needs to be loaded, it calls the DashIndexHandler.• When playing, getNextFile is called• When seeking, getFileForTime is called• Both return an HTTPStreamRequest, which OSMF uses to request the proper segment
  31. 31. DashFileHandler• When the requested file is returned, it is handed to the DashFileHandler for processing – initializeProcessing – gets decoder – processFileSegment – asks decoder to process data• We are using a Decoder called MP4Decoder• MP4Decoder returns FLVBytes for the data within the MDAT• processFileSegment returns ByteArray (FLVBytes) to OSMF, for adding to the NetStream buffer
  32. 32. MP4Decoder• processData() – iterates through all boxes in segment – Calls getNextBox()• getNextBox knows how to process each specific box type• When processing MDAT, individual samples are wrapped with FLVBytes• FLVBytes wrap specific binary header around data, as required by NetStream.appendBytes()
  33. 33. Digital Primates DASH Player• Currently our DASH player is not commercially available• We use it as the basis for DASH players we build for our clients• Initially built as a learning exercise to better understand DASH
  34. 34. Current DP Dash Player• We currently support SegmentLists, SegmentTemplates and SegmentBase in manifest – Byte Range Requests only currently supported via Akamai• We support Dynamic Adaptive Streaming for VOD Streams• We have tested with streams up to 1080p• We have tested with streams from several different encoders
  35. 35. DP Dash Player plans• 4Q12 • 1Q13 – Basic Seek Fixes – Initial JavaScript Player – Improved multi-channel – Support for DRM audio support – Support for Enhanced seeking – Support for Live • 2Q13 Streams – Investigate HEVC (h.265) – Continue to test and fix streams from more – Initial Android Player encoding partners • 3Q13 – Initial iOS Player
  36. 36. Resources• GPAC – http://gpac.wp.mines-telecom.fr – Provides baseline test streams – Provides baseline player• MP4Parser – http://code.google.com/p/mp4parser/ – Open Source java project – Allows for display of contents within boxes• DASH Industry Forum – http://www.dashif.org
  37. 37. Questions?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×