This document provides an overview of Apple's HTTP Live Streaming (HLS) protocol for dynamically adapting video streaming to network conditions. It describes the basics of HLS including how content is prepared and served, how clients play the stream by obtaining playlist files that list available media segments, and new tags defined by the HLS protocol such as EXT-X-TARGETDURATION and EXT-X-MEDIA-SEQUENCE. It also compares HLS to other adaptive streaming protocols and shows examples of analyzing an HLS stream with Wireshark.
HLS is a popular streaming protocol that is used to transmit video and audio content from a server to a user via the internet. Here’s how it works: http://bit.ly/2IGBKy1
HLS is a popular streaming protocol that is used to transmit video and audio content from a server to a user via the internet. Here’s how it works: http://bit.ly/2IGBKy1
Intro to Single / Two Rate Three Color Marker (srTCM / trTCM)Kentaro Ebisawa
The objective of this document is to provider entry point for people to understand srTCM and trTCM (single / two rate Tree Color Marker).
This document will explain algorithm described in below RFCs, and compare two different algorithm used for trTCM.
RFC 2697 - A Single Rate Three Color Marker
RFC 2698 - A Two Rate Three Color Marker
RFC 4115 - A Differentiated Service Two-Rate, Three-Color Marker with Efficient Handling of in-Profile Traffic
Our presentation from the media web symposium 2013 in Berlin on the open source landscape around MPEG-DASH as well as on cloud-based services for MPEG-DASH
All the content of this website is informative and non-commercial, does not imply a commitment to develop, launch or schedule delivery of any feature or functionality, should not rely on it in making decisions, incorporate or take it as a reference in a contract or academic matters. Likewise, the use, distribution and reproduction by any means, in whole or in part, without the authorization of the author and / or third-party copyright holders, as applicable, is prohibited.
Video coding is an essential component of video streaming, digital TV, video chat and many other technologies. This presentation, an invited lecture to the US Patent and Trade Mark Office, describes some of the key developments in the history of video coding.
Many of the components of present-day video codecs were originally developed before 1990. From 1990 onwards, developments in video coding were closely associated with industry standards such as MPEG-2, H.264 and H.265/HEVC.
The presentation covers:
- Basic concepts of video coding
- Fundamental inventions prior to 1990
- Industry standards from 1990 to 2014
- Video coding patents and patent pools.
Intro to InfluxDB 2.0 and Your First Flux Query by Sonia GuptaInfluxData
In this InfluxDays NYC 2019 talk, InfluxData Developer Advocate Sonia Gupta will provide an introduction to InfluxDB 2.0 and a review of the new features. She will demonstrate how to install it, insert data, and build your first Flux query.
Intro to Single / Two Rate Three Color Marker (srTCM / trTCM)Kentaro Ebisawa
The objective of this document is to provider entry point for people to understand srTCM and trTCM (single / two rate Tree Color Marker).
This document will explain algorithm described in below RFCs, and compare two different algorithm used for trTCM.
RFC 2697 - A Single Rate Three Color Marker
RFC 2698 - A Two Rate Three Color Marker
RFC 4115 - A Differentiated Service Two-Rate, Three-Color Marker with Efficient Handling of in-Profile Traffic
Our presentation from the media web symposium 2013 in Berlin on the open source landscape around MPEG-DASH as well as on cloud-based services for MPEG-DASH
All the content of this website is informative and non-commercial, does not imply a commitment to develop, launch or schedule delivery of any feature or functionality, should not rely on it in making decisions, incorporate or take it as a reference in a contract or academic matters. Likewise, the use, distribution and reproduction by any means, in whole or in part, without the authorization of the author and / or third-party copyright holders, as applicable, is prohibited.
Video coding is an essential component of video streaming, digital TV, video chat and many other technologies. This presentation, an invited lecture to the US Patent and Trade Mark Office, describes some of the key developments in the history of video coding.
Many of the components of present-day video codecs were originally developed before 1990. From 1990 onwards, developments in video coding were closely associated with industry standards such as MPEG-2, H.264 and H.265/HEVC.
The presentation covers:
- Basic concepts of video coding
- Fundamental inventions prior to 1990
- Industry standards from 1990 to 2014
- Video coding patents and patent pools.
Intro to InfluxDB 2.0 and Your First Flux Query by Sonia GuptaInfluxData
In this InfluxDays NYC 2019 talk, InfluxData Developer Advocate Sonia Gupta will provide an introduction to InfluxDB 2.0 and a review of the new features. She will demonstrate how to install it, insert data, and build your first Flux query.
Windows Azure Media Services June 2013 updateMingfei Yan
This is a update presentation for Windows Azure Media Services June 2013 update. It showcases dynamic packaging, MPEG-DASH release and Live streaming sneak peek. You could view session video here: https://channel9.msdn.com/Events/Build/2013/3-549
Final Year Project
> Creating a Video Player with Python
> Used Software - Pycharm & Pyhton 3.6
> Can play any Video , mp3, mp4,
> Features:
+ Play
+ Pause
+ Next
+ Previous
+ Volume contol
Alto is a game audio localization tool that compares thousands of dialogues lines in different languages and generates reports. It checks their audio format, loudness, duration and more. It can also interface with game audio middleware (Adx2, Fabric, Fmod, Wwise...). It includes special tools such as an interactive dialogue tester, a speech synthesizer for placeholder dialogue and more!
http://www.tsugi-studio.com/?page_id=1923
Locating objects in images (“detection”) quickly and efficiently enables object tracking and counting applications on embedded visual sensors (fixed and mobile). By 2012, progress on techniques for detecting objects in images – a topic of perennial interest in computer vision – had plateaued, and techniques based on histogram of oriented gradients (HOG) were state of the art. Soon, though, convolutional neural networks (CNNs), in addition to classifying objects, were also beginning to become effective at simultaneously detecting objects. Research in CNN-based object detection was jump-started by the groundbreaking region-based CNN (R-CNN). We’ll follow the evolution of neural network algorithms for object detection, starting with R-CNN and proceeding to Fast R-CNN, Faster R-CNN, “You Only Look Once” (YOLO), and up to the latest Single Shot Multibox detector. In this talk, we’ll examine the successive innovations in performance and accuracy embodied in these algorithms – which is a good way to understand the insights behind effective neural-network-based object localization. We’ll also contrast bounding-box approaches with pixel-level segmentation approaches and present pros and cons.
Welcome to a primer on the back-propagation (of errors) as it applies to the training of neural networks. We answer the question, what's the contribution of the back-propagation-technique?
1. (Apple) HTTP Live Streaming
(HLS)
Dynamically adapting video
streaming to network conditions
Auro Tripathy
2. Outline
• The basics of adaptive streaming
• A Wireshark view
• New tags defined by the HLS protocol
• A comparison with other adaptive
streaming protocols
November 8, 2011 Auro 2
3. HLS Streaming Architecture
Content Preparation
HTTP Content Server
Media Media
Encoder Segmenter Index HTTP
File Cache
Segment
November 8, 2011 Auro 3
4. The Basics of HLS
• Protocol for transmitting alternate versions
(i.e., bitrates) of the stream
• Media can be served soon after its created
– Not actually “live”, but near real-time
• Specify a presentation with the URI of a
playlist file
• A playlist file is an ordered list of media URI
and info tags
• Each media URI refers to a media file which
is a segment of a single contiguous file
November 8, 2011 Auro 4
5. How does the client play the
stream?
• Client first obtains the playlist file
– Initial playlist file may contain alternates (see
next slide)
• Client decides which alternate playlist file
to play
• Gets and plays each media file in the
playlist.
November 8, 2011 Auro 5
6. Server Provides Alternate
Playlists to Clients
Server
Alternate 1
Index
File TS Segment
Index Alternate 2
File Index
w/alternates File TS Segment
Alternate 3
Index
File TS Segment
November 8, 2011 Auro 6
7. M3U format – the History
• M3U stands for MP3 URL.
• Files that contain the .M3U file-extension are
MP3 playlist files
– commonly referred to as Winamp playlist files
• Important to remember, this file does not
contain media, only references to media
• Files whose name end with .m3u are
encoded with US-ASCII.
• The unicode version of "m3u" is "m3u8",
which uses UTF-8 unicode characters
November 8, 2011 Auro 7
8. Extended M3U
• Defines two tags: EXTM3U & EXTINF
• First line must be #EXTM3U
– This distinguishes it from a M3U file
• EXTINF is a record marker that describes the
media file identified by the URI that follows it.
• Example
#EXTM3U
#EXTINF:<duration>, <title>
http://media.example.com/entire.ts
November 8, 2011 Auro 8
9. Server Tasks to Achieve HLS
• Encode video in H.264 format and Single
Program Transport Stream “container”
– Encode in different bitrates
– taken from live feed or from a file
• Use a stream segmenter to generate short
“chunks” of content – typically 10 seconds
each - and generate a playlist file (m3u or
m3u8) indicating from where to download the
chunks
• Serve via HTTP server, and provide
appropriate caching for performance
November 8, 2011 Auro 9
10. Who supports HLS?
• Apple, of course
– Authored the protocol
http://tools.ietf.org/html/draft-pantos-http-live-
streaming-07
• Support available in Android 3.0
– http://developer.android.com/sdk/android-3.0-
highlights.html
November 8, 2011 Auro 10
12. Server Offering Multiple Playlist Files
Different encodings of the same presentation
Client request
Server offers four
alternative playlists at
200 Kbps, 311.1 Kbps,
484.4 Kbps & 737.8 Kbps
November 8, 2011 Auro 12
13. Client fetches the first Playlist File
Client request 200 Kbps
playlist
Duration of segment in
seconds; appears once
Segment URLs
November 8, 2011 Auro 13
14. Note - Playlist file ends with a #EXT-X-ENDLIST
Indicates no more media files
will be added to the Playlist file
November 8, 2011 Auro 14
19. Summarizing the Sequence
Server Client
Respond with four alternate playlists
Location of all segments in Playlist
Location of all segments in Playlist
Respond with A/V data
November 8, 2011 Auro 19
20. VLC Plays HLS Test File
http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8
November 8, 2011 Auro 20
22. HLS defines the following new tags
Each with it own attribute list
• EXT-X-TARGETDURATION
• EXT-X-MEDIA-SEQUENCE
• EXT-X-KEY
• EXT-X-PROGRAM-DATE-TIME
• EXT-X-ALLOW-CACHE
• EXT-X-PLAYLIST-TYPE
• EXT-X-STREAM-INF
• EXT-X-ENDLIST
• EXT-X-DISCONTINUITY
• EXT-X-VERSION
November 8, 2011 Auro 22
23. Tag – EXT-X-TARGETDURATION
• EXT-X-TARGETDURATION:<s>
– Specifies the maximum media file duration
– Appears only once in the playlist file
– <s> is an integer indicating the duration in
seconds
November 8, 2011 Auro 23
24. Tag – EXT-X-MEDIA-SEQUENCE
• Each media file URI in a playlist must have
a unique integer number
• Sequence # := Sequence # of URI that
preceded it + 1
• Tag indicates the sequence number of the
first URI that appears in the playlist file
• Format
EXT-X-MEDIA-SEQUENCE:<number>
November 8, 2011 Auro 24
25. Tag – EXT-X-KEY
• Contains information to decrypt media files that
follow it
• Format
#EXT-X-KEY:<attribute-list>
• METHOD attribute specifies encryption method
(AES-128 or NONE)
• URI attribute identifies the Key file
– Key file contains the cipher key to decrypt subsequent
media files in the Playlist
– Quoted string with URI
• IV attribute specifies the initialization vector to be
used with the key
November 8, 2011 Auro 25
26. Tag – EXT-X-PROGRAM-DATE-
TIME
• Associated with the next media file
• Should indicate a time zone
• Format
EXT-X-PROGRAM-DATE-TIME:<YYYY-MM-DDThh:mm:ssZone>
November 8, 2011 Auro 26
27. Tag – EXT-X-ALLOW-CACHE
• Indicates whether a client MAY or MUST-
NOT cache downloaded media files for
later playback.
• Tag applies to all segments in the playlist
• Format
#EXT-X-ALLOW-CACHE:<YES|NO>
November 8, 2011 Auro 27
28. Tag - EXT-X-PLAYLIST-TYPE
• Optional tag provides mutability
information about the playlist file
• Format:
#EXT-X-PLAYLIST-FILE:<Event|VoD>
November 8, 2011 Auro 28
29. Tag – EXT-X-ENDLIST
• Indicates no more media files will be
added to the Playlist file
• Occurs only once
• Format:
#EXT-X-ENDLIST
November 8, 2011 Auro 29
30. Tag – EXT-X-STREAM-INF
• Tag indicates that the next URI in the playlist
file indentifies another playlist file
• Format/example
#EXT-X-STREAM-INF:<attribute-list>
<URI of playlist file>
• Defined attributes
– BANDWIDTH
– PROGRAM-ID
– CODECS
– RESOLUTION
November 8, 2011 Auro 30
31. Tag – EXT-X-DISCONTINUITY
• Tag indicates an encoding discontinuity
between the media file that follows it and the
one that preceded it
• Reason
– File format change
– Number and type of tracks
– Encoding parameters
– Encoding sequence
– Timestamp sequence
• Format
– #EXT-X-DISCONTINUITY
November 8, 2011 Auro 31
32. Tag – EXT-X-VERSION
• Tag indicates the compatibility version of
the Playlist file
• One occurrence
• Format
#EXT-X-VERSION:<n>
November 8, 2011 Auro 32
33. Media Files
• Media files are formatted as MPEG-2
transport streams
• It’s a singe program transport stream
• Should have a PAT/PMT at the start of
each files
• Client should be able to handle any
encoding parameter change such as a
resolution change
November 8, 2011 Auro 33
34. Key Files
• The EXT-X-KEY tag has the URI and the
IV attributes to identify the key file and the
Initialization Vector
• If the IV is not present, client must use the
sequence number of the media file as the
IV
• Note, varying the IV increases the strength
of the cipher
November 8, 2011 Auro 34
35. Conclusion – Lots of upside with
the protocol
• Uses the HTTP protocol
– Firewall-friendly
– HTTP server are ubiquitous thus reducing
operational complexity
– Unifies the delivery for three screens
(TV/Mobile/Tablet)
– Scales, no session state on the HTTP server
• Chunked transfers allow the client to
compensate/optimize in near real-time for
varying network conditions
November 8, 2011 Auro 35
36. Conclusion – Potential
Downsides
• Choice of MPEG-2 single program
transport streams
– 188 byte packet structure with 4-byte header
creates non-negligible overhead
• VCR-like trick-modes (seek to a time
forward or backward) can cause
noticeable delays if the segment size is
large
November 8, 2011 Auro 36