The document discusses the migration from Flash to HTML5 for streaming media, highlighting key considerations such as API integrations, encoding formats, player choices, and DRM requirements. It emphasizes the need for careful planning in transitioning to modern player architectures and adapting to various mobile and set-top box environments. Additionally, the document provides examples and strategies for successful implementation while addressing potential pitfalls in the production process.

1. Transitioning from Flash to HTML5
Streaming Media East – Streaming Media University
Monday, May 15, 2017
9:30 am to 12:30 pm

2. ● Pioneers in hybrid video delivery systems to accompany growth in OTT traffic
● Experts in HTML5 video through integrations into open-source & proprietary
players
● Consultants in the transition from Flash to HTML5
Streamroot: Who are we?

3. And who are you?
Infinite scale, limitless delivery.

4. I. HTML5 basics
A. Why make the transition
B. APIs: the basics
II. Delving into the migration
A. Support
B. Encoding
C. Formats
D. Players
E. DRMs, ads and business features
F. HTTPs
III. Choosing a web player
A. Modern player architecture
B. Considerations & what to look for
C. Benchmark of open-source and proprietary options
What we’ll be talking about today.
Infinite scale, limitless delivery.

5. IV. Choosing mobile & STB
Environments
Building blocks
Your player options
Native vs Webview
V. Examples: hls.js & dash.js
Debugging, subtitles, encryption, ads, Streamroot, ABR
VI. Going from POC to production
A. What could go wrong
B. What happens after I do a POC? Workflow, fallbacks, etc.
C. Smoothly transitioning into production: QoS metrics, AB testing
D. Useful tools & links
What we’ll be talking about today.
Infinite scale, limitless delivery.

6. Infinite scale, limitless delivery.
I. It’s time to switch to HTML5.

7. Infinite scale, limitless delivery.
I. HTML5 Basics
APIs
1. Media Source Extensions
• Makes HTTP adaptive streaming
possible in HTML5
• Use JavaScript to build streams and
inject data into the video tag’s buffer
Source: W3C specs

8. Infinite scale, limitless delivery.
I. HTML5 Basics
APIs
2. Encrypted Media Extensions
• Allows for DRMs in HTML5
• Provides a way to interact with
content protection systems
• Plugin-free
• Common Encryption (CENC):
standardized key and encryption
methods - multiple DRMs for same
file
Source: W3C specs

9. Infinite scale, limitless delivery.
II. Delving into the migration
Consider every part of your workflow:
- MSE browser & device support
- Encoding
- Format
- Players
- DRMs
- Ads and other business features
- HTTPS?

10. Infinite scale, limitless delivery.
Source: Bitmovin
II. Delving into the migration
HTML5 Format Support

11. Infinite scale, limitless delivery.
II. Delving into the migration
Encoding
Significant changes on encoding side that
can take some time. Options:
- H.264/AVC → HEVC (H.265)
- VP9 → AV1
Either way:
- Encode to MP4 and then repackage as
necessary for target platforms
- Many vendors out there: Wowza,
Unified Streaming Platform, etc.
Source: Encoding.com 2017 Global
Media Format Report

12. Infinite scale, limitless delivery.
HDS
II. Delving into the migration
Packaging & Formats
Source: Encoding.com 2017 Global
Media Format Report
72%
21%
7%

13. Infinite scale, limitless delivery.
I. HTML5 Basics
Formats available
Source: Encoding.com

14. Infinite scale, limitless delivery.
II. Delving into the migration
Formats: CMAF
Common Media Application Format:
- ISOBMFF + CENC + H.264/HEVC/etc. + Dash OR Hls manifest
- Apple supports fragmented MP4 in latest HLS version on iOS 10
PROs
Play DASH and HLS from single set of source files!
- Cut storage costs in half, efficient caching
- fMP4 involves less overhead than mpeg-ts
- No more need to transmux
- Single media engine for HLS and DASH
WARNING
- Legacy devices
- DRMs

15. Infinite scale, limitless delivery.
II. Delving into the migration
Formats: CMAF

16. Infinite scale, limitless delivery.
II. Delving into the migration
Standardization efforts

17. Infinite scale, limitless delivery.
II. Delving into the migration
DRM
Switch from token authorization to HTML5 DRM
- Microsoft: PlayReady -> Dash/fMP4 -> CTR
- Chrome, FF: Widevine -> Dash/fMP4 -> CTR, CBC
- iOS: FairPlay -> HLS/TS -> CBC
Keep in mind Flash fallbacks.
Mobile and STB also have DRM capabilities.

18. Infinite scale, limitless delivery.
II. Delving into the migration
DRM
For complete compatibility lists, including mobile and STB:
https://www.buydrm.com/keyos-platform-compatibility-matrix

19. Infinite scale, limitless delivery.
II. Delving into the migration
Advertising
You’ll need to rewrite the client-side Ad selection in HTML5
Good news: Flash ads are dying too!
On Google Ad-exchange:
- No video ads can be added in Flash since April 2017
- No video ads will be served in Flash starting July 2017
But because of legacy, still need some Flash fallbacks...

20. Infinite scale, limitless delivery.
II. Delving into the migration
Server-side ad insertion (SSAI)
A new approach on the market: SSAI.
PROs
- Not blocked by ad-blockers
- A smoother experience
CONs
- possible caching challenges
- not all players are ready

21. Infinite scale, limitless delivery.
II. Delving into the migration
HTTPS
Beware!
Playing HTTP streams even in an HTTPS environment is not an option in HTML5.
Changing to HTTPs can be expensive and long, depending on your CDN.

22. Infinite scale, limitless delivery.
III. Player architecture
Modern player architecture
Before choosing:
1. Understand the specificities of your use case: type of streaming, user habits,
target platforms, etc.
2. Know your feature requirements: ads, paywall, UX, channels, playlists,
analytics, etc.
More flexible architecture with separation between UX and media engine logic!

23. Infinite scale, limitless delivery.
It’s like...
III. Player architecture
Modern player architecture

24. … your favorite burger.
Infinite scale, limitless delivery.
It’s like...
III. Player architecture
Modern player architecture

25. Infinite scale, limitless delivery.
UI
III. Player architecture

26. Infinite scale, limitless delivery.
Skin
UI
III. Player architecture

27. Infinite scale, limitless delivery.
Skin
Ads
Authentication
UI
III. Player architecture

28. Infinite scale, limitless delivery.
Skin
Ads
Playlists
Authentication
UI
Social sharing
III. Player architecture

29. Infinite scale, limitless delivery.
Skin
Ads
Playlists
Authentication
UI
Media Engine(s)
Social sharing
III. Player architecture

30. Infinite scale, limitless delivery.
Skin
Ads
Playlists
Authentication
Media Engine(s)
UI
Media Engine(s)
Social sharing
III. Player architecture

31. Infinite scale, limitless delivery.
Skin
Ads
Playlists
Authentication
Social sharing
Media Engine(s)
UI
Media Engine(s)
Playback
& DRM
III. Player architecture

32. Infinite scale, limitless delivery.
Skin
Ads
Playlists
Authentication
Content
decryption
module
Media Engine(s)
DRM
Manager
UI
Media Engine(s)
Playback
& DRM
Social sharing
III. Player architecture

33. Infinite scale, limitless delivery.
Skin
Ads
Playlists
DRM
Manager
Decoder /
Renderer
Authentication
Content
decryption
module
Media Engine(s)
UI
Media Engine(s)
Playback
& DRM
Social sharing
III. Player architecture

34. Infinite scale, limitless delivery.
1. Skin - the graphic design
of your player
III. Player architecture
User Interface

35. Infinite scale, limitless delivery.
1. Skin - the graphic design
of your player
2. UI logic - features defining
all interaction with the user
on top of video playback
Lots of plugins available, video.js for
example
III. Player architecture
User Interface

36. Infinite scale, limitless delivery.
3. Business logic
Authentication
Payment
Ads
*Configuration / device detection logic!
III. Player architecture
User Interface

37. Infinite scale, limitless delivery.
Sample UI workflow with authentication, channels and pre-roll ad
III. Player architecture
User Interface

38. Infinite scale, limitless delivery.
Easily customizable, with many plugins available for use or inspiration.
Add features as plugins/modules to the core UI base.
Create a unified user experience across browsers, even if the media engine behind it
may vary from device to device.
Check out tools such as React native, Haxe
III. Player architecture
User Interface

39. Infinite scale, limitless delivery.
III. Player architecture
Media Engine

40. Infinite scale, limitless delivery.
III. Player architecture
Media Engine

41. Infinite scale, limitless delivery.
III. Player architecture
Media Engine

42. Infinite scale, limitless delivery.
III. Player architecture
Media Engine

43. Infinite scale, limitless delivery.
III. Player architecture
Media Engine

44. Infinite scale, limitless delivery.
III. Player architecture
Media Engine

45. Infinite scale, limitless delivery.
Extremely important yet too often neglected.
Most often necessary to have several media engines to reach your audience.
III. Player architecture
Media Engine

46. Infinite scale, limitless delivery.
III. Player architecture
Decoder & DRM manager

47. 1. General criteria
- device, format and codec dependencies
Infinite scale, limitless delivery.
III. What to look for
2. Custom features
- DRMs
- DVR subtitles
- QoS, etc.
3. Perfs & Quality
- startup time
- ABR
- error strategies
4. Street cred, robustness, ease of use

48. Light-touch dev:
- Simplicity and stability of the media engine
- Assess extensibility & ease of incorporating special features
- Media engine customization
- Events exposed
- Debugging
Infinite scale, limitless delivery.
Digging deeper:
- Robust yet flexible core design
- Tests and testing coverage
- community & support
III. What to look for

49. Infinite scale, limitless delivery.
III. What’s out there?
Disclaimer!
- What follows are all GOOD options.
- We’ve tried to be solely objective based on our research and experience.
- Features, support and upkeep are always changing.

50. Infinite scale, limitless delivery.
PROs
Supported by DASH-IF
Pushed & maintained by Akamai + tier-1s (BBC)
Highest visibility, big community
Lots of features and use-cases handled
Widely used in production
Huge test suite + online test page
CONs
A little bit of technical debt and complexity
Non-trivial API and customization config
H264/AAC support only
III. What’s out there?

51. Infinite scale, limitless delivery.
PROs
Built by an entire at Google: solid & smart
Performances and robustness, quick to improve
Simple to get started, good tutorials
Only one supporting WebM, VP8, VP9, open audio codecs
Good support on github and Google groups
CONs
Google-centric
Today lacks some features for large broadcasters
Stricter PR & features policy
No ES6 support
Fewer OVP and open-source all-in-one integrations
III. What’s out there?

52. Infinite scale, limitless delivery.
PROs
Created by Dailymotion, built from scratch by author of Flashls
Enormous traction and visibility
Clear architecture design, easily extendable
Good robustness and debug demo, responsive support
Widely used in prod by all-in-one players + tier-1s
CONs
Some restrictions from the transmuxing & HLS: no DRMs today,
only AES128
III. What’s out there?

53. Infinite scale, limitless delivery.
PROs
Seamless Flash fallback with a MediaSource polyfill
De facto solution for HLS with VideoJS
Large community of users (Brightcove + Videojs)
In production with Brightcove with a wide range of customers
CONs
Videojs plugin: not usable without videojs
Lack of public debug tools or pages
Learning curve on providers & tech behind it
III. What’s out there?

54. But there are also a lot of off-the-shelf options.
Infinite scale, limitless delivery.
III. What’s out there?

55. Different environments:
- Mobile web (Safari on iOS, Chrome on Android)
- Native Apps
Different APIs, different access levels… often more restrictive than web!
Infinite scale, limitless delivery.
IV. Mobile and other platforms

56. Infinite scale, limitless delivery.
IV. Mobile and other platforms
Building blocks: iOS
Base player: AV Player
No low level-APIs
HLS
Fairplay
Browsers: no MSE on Safari, but native HLS support

57. -
Infinite scale, limitless delivery.
IV. Mobile and other platforms
Building blocks: Android
Base Player: MediaPlayer
Low level APIs:
- MediaCodec (like MSE)
- MediaDRM (like EME)
Extended open-source framework: ExoPlayer
DASH, HLS, mp4, Smooth...
Widewine
Browsers: as on Desktop, supports MSE!

58. -
Infinite scale, limitless delivery.
IV. Mobile and other platforms
Building blocks: Other devices
TVos: like iOS
Android TV: like Android + Playready
STBs:
- Android TV based
- Chromium/Opera Middle-ware based
- Proprietary frameworks (legacy)
SmartTVs:
- big fragmentation (Android, Tizen, proprietary…)
- Not following standards very well

59. -
Infinite scale, limitless delivery.
IV. Mobile and other platforms
What players to choose?
Build yourself:
- AV Player on iOS
- Exoplayer on Android
Off-the-shelf:
- all built on AV Player & ExoPlayer
- unified UX, UI & config
- adding some bonus features
(simpler DRMs, offline storage, etc)

60. -
Infinite scale, limitless delivery.
IV. Mobile and other platforms
HTML5 approach?
iOS:
- no MSE in Safari, nor in Webview
Android:
- can use MSE-based players directly with a WebView!
PROs
Same code and logic on Desktop & Mobile
Comparable performance
Easier UX & UI: HTML & CSS
CONs
Less options than with ExoPlayer
Need to know JS & HTML5

61. -
Infinite scale, limitless delivery.
IV. Mobile and other platforms
DRMs
iOS:
- Fairplay, only on the latest versions
Android:
- Widewine
So if you are using DRMs, you will need at least Widewine, Fairplay & Playready to
have a good devices support.

62. All available online on github:
https://github.com/streamroot/benchmarking-player
Contains:
Media engines code
Media samples
Examples
Infinite scale, limitless delivery.
V. Examples

63. MSE example
https://github.com/streamroot/benchmarking-player/blob/master/samples/mse.html
Infinite scale, limitless delivery.
V. Examples

64. Hls.js examples
https://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs.html
https://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs-AES.html
https://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs-subtitles.html
https://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs-ads.html
https://github.com/streamroot/benchmarking-player/blob/master/samples/hls.js/hlsjs-streamroot.ht
ml
Online debug demo: debug http://dailymotion.github.io/hls.js/demo/
Videojs Ads plugin: https://github.com/videojs/videojs-contrib-ads
Ads state diagram: https://raw.githubusercontent.com/videojs/videojs-contrib-ads/master/ad-states.png
Hls.js Events: https://github.com/dailymotion/hls.js/blob/master/API.md#runtime-events
Video Events simulator: http://docs.youbora.com/video-event-simulator/
Streamroot Clappr integration: https://github.com/streamroot/clappr-p2phls-plugin
Strearmoot Videojs integration: https://github.com/streamroot/videojs5-hlsjs-p2p-source-handler
Infinite scale, limitless delivery.
V. Examples

65. Dash.js examples
https://github.com/streamroot/benchmarking-player/blob/master/samples/dash.js/dashjs.html
https://github.com/streamroot/benchmarking-player/blob/master/samples/dash.js/dashjs-widewine.ht
ml
https://github.com/streamroot/benchmarking-player/blob/master/samples/dash.js/dashjs-subtitles.ht
ml
https://github.com/streamroot/benchmarking-player/blob/master/samples/dash.js/dashjs-ads.html
https://github.com/streamroot/benchmarking-player/blob/master/samples/dash.js/dashjs-streamroot.
html
Online debug page:
http://dashif.org/reference/players/javascript/v2.3.0/samples/dash-if-reference-player/index.html
Dash.js metrics & events: http://cdn.dashjs.org/latest/jsdoc/module-DashMetrics.html
Dash.js ABR logic: https://github.com/Dash-Industry-Forum/dash.js/wiki/ABR-Logic
Streamroot Videojs integration: https://github.com/streamroot/videojs5-dashjs-p2p-source-handler
Infinite scale, limitless delivery.
V. Examples: Dash.js

66. Other media engines
https://github.com/streamroot/benchmarking-player/blob/master/samples/shaka.html
https://github.com/streamroot/benchmarking-player/blob/master/samples/videojs.hls.html
Infinite scale, limitless delivery.
V. Examples

67. What could go wrong?
- - Encoding
- - DRMs
- - HTTPS
Infinite scale, limitless delivery.
V. The Path to Production!

68. I did a POC. Now what?
- - Do you need a Flash Fallback?
- - Moving the UX to HTML5
- - Build the media engine extendable architecture (providers)
- - (optional) choose a Fallback alternative
https://github.com/streamroot/videojs5-hlsjs-source-handler/blob/master/lib/videojs5-hlsjs-source-handler.js
https://github.com/videojs/video.js/blob/66922a818e588b8954989c439e1be790777966ae/docs/guides/tech.md
https://github.com/faisalman/ua-parser-js
Infinite scale, limitless delivery.
V. The Path to Production!

69. Making a smooth transition.
- - HTML5 player rollout
- - Playback & QoS Metrics
- - AB Testing!
Infinite scale, limitless delivery.
V. The Path to Production!

70. Infinite scale, limitless delivery.
V. The Path to Production!
AB testing pipeline: General setup

71. Client AB testing methods:
1) Dynamic Config Injection
2) Different builds with a reverse proxy
Infinite scale, limitless delivery.
V. The Path to Production!
AB testing pipeline: General setup

72. How much time does it take?
The build vs. buy tradeoff.
- Do I have the internal resources to develop,
integrate, customize and maintain?
- Are the features I need already available in
off-the-shelf and/or open-source solutions?
Consider time to market.
Infinite scale, limitless delivery.
V. The Path to Production!

73. Questions?
Infinite scale, limitless delivery.

74. Nikolay Rodionov, Co-Founder and CPO, nikolay@streamroot.io
Erica Beavers, Head of Partnerships, erica@streamroot.io
Infinite scale, limitless delivery.
Get in touch!