Live streaming of video and subtitles using HTTP and Web Browser

1. Live HTTP Streaming of Video and
Subtitles within a Browser
Cyril Concolato
Jean Le Feuvre
{concolato, lefeuvre} @ telecom-paristech.fr
MMSys 2013

2. Context
1
■ Video Streaming is popular on the Web
• With subtitle support in VOD
• With live streaming solutions (DASH, HLS, …)
■ What about live subtitles ?
• Several specifications being developed
− ISOBMFF Amendment on Timed Text and Timed
Overlays, for use with DASH
− WebVTT draft (used in HLS v10)
• No deployment yet within browsers
■ Can we today stream and play live video+subtitles
within a browser?
• Yes !
• Development of a Proof-of-Concept
Live HTTP Streaming of Video and Subtitles within a Browser
- MMSys Demo 2013

3. Architecture
3
Live HTTP Streaming of Video and Subtitles within a Browser
- MMSys Demo 2013

4. Demonstration Media Formats
■ The ISO Base Media File Format
• To carry video stream segments
• Conformant to MPEG DASH and Google Chrome restrictions
■ The WebVTT format
• To carry subtitle data
− as separate text files (not multiplexed in the video container)
− Similar to HLS v10, with some differences (no embedded
video timing)
• Generation of non-overlapping, constant duration segments
(compatible with MPEG DASH)
■ MPEG DASH MPD
• To represent a dynamic live/long-running streaming service
• To enable synchronization of the video and subtitle streams.
4
Live HTTP Streaming of Video and Subtitles within a Browser
- MMSys Demo 2013

5. Demonstration software components
■ GPAC MP4Box
• To produce video segments on-the-fly and delete old
ones
− From a pre-recorded file (looped indefinitely)
■ VTTLiveGenerator
• Ad-hoc tool
• To produce subtitle segments on-the-fly, synchronized
with the video and delete old ones
■ Google Chrome
• Canary version
• To display the content using HTML5 and JavaScript
5
Live HTTP Streaming of Video and Subtitles within a Browser
- MMSys Demo 2013

6. Usage of MP4Box
■ Fragmenting a file repeatedly using -dash-ctx option
MP4Box.exe -dash-live 2000 -rap
-dash-ctx dash-live.txt
-segment-name counter-live
counter.mp4
■ Example of MP4Box’s DASH context
[DASH]
InitializationSegment=counter_liveinit.mp4
BitstreamSwitching=yes
InitializationSegmentSize=860
MaxSegmentDuration=1.000000
NextSegmentIndex=11
NextFragmentIndex=11
PeriodDuration=9.960000
[TrackID_1]
NextDecodingTime=249000
6
Live HTTP Streaming of Video and Subtitles within a Browser
- MMSys Demo 2013

7. Example of Video Content
7
Live HTTP Streaming of Video and Subtitles within a Browser
- MMSys Demo 2013

8. Example of WebVTT Segment
■ Each segment is a complete WebVTT file with multiple
cues,
• changing header, cue indices, and timing values
WEBVTT Segment 6
61
00:00:12.200 --> 00:00:12.400
This is cue 61 (start: 00:00:12.200 -- end:
00:00:12.400)
...
70
00:00:14.000 --> 00:00:14.200
This is cue 70 (start: 00:00:14.000 -- end:
00:00:14.200)
8
Live HTTP Streaming of Video and Subtitles within a Browser
- MMSys Demo 2013

9. Example of DASH Description
■ Use of a “dynamic” MPD
• No minUpdate attribute
• Without updates (assuming known/static timing)
■ A single period with 2 adaptation sets
• 1 video representation
• 1 subtitle representation
• Same video and subtitle segments duration (easy)
■ live-vtt-mp4.mpd
9
Live HTTP Streaming of Video and Subtitles within a Browser
- MMSys Demo 2013

10. Rendering using Google Chrome Canary
■ HTML page
• Use of HTML5 <video> element with an HTML
<track> element for playback synchronization
• Use of JavaScript code to control the download logic
■ JavaScript algorithm
• Fetch and parse the MPD,
• Create the <video> and <track> elements,
• Determine and fetch the video and subtitles segments
using the XMLHttpRequest API,
• Feed the video segments to the media decoder engine
using the Media Source extension API,
• Load each subtitle segment using an HTML5 work-
around
10
Live HTTP Streaming of Video and Subtitles within a Browser
- MMSys Demo 2013

11. Demo
11
Live HTTP Streaming of Video and Subtitles within a Browser
- MMSys Demo 2013

12. Discussion/Limitations
■ Working demo
• Supports time shifting or not
• Needs better integration of video & subtitle MPDs at server side
• Needs at client side:
− Better MPD parsing
− Better simple segment index algorithm
■ Issues
• Rendering performance problem if more than 4 cues per second
• User Interface problem
− Unusable built-in UI for long-running live events
− Needs custom UI to disable seeking before the first
received data
• HTML 5 limitations to load cues
− No MSE equivalent for cues (outside of VTT in MP4)
− No easy method to parse cues
12
Live HTTP Streaming of Video and Subtitles within a Browser
- MMSys Demo 2013

13. Questions ?
Come and see me