Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introduce native html5 streaming player

1,403 views

Published on

How did I create a player support:
- adaptive streaming
- DRM

Also introduce DIJON.js, a dependency management framework of javascript

Published in: Engineering

Introduce native html5 streaming player

  1. 1. HOW DID I CREATE A HTML5 VIDEO STREAMING PLAYER and what I’ve learned KKBOXVideo 陳建辰 jessechen
  2. 2. AGENDA • Some beforehand knowhow • Why developing this player • HTML5 extensions • Introduce our player • Future of this player
  3. 3. SOME BEFOREHAND KNOWHOW
  4. 4. analog-to- digital adapter digital raw data PROCESS OF PLAYINGVIDEO 1. Make it digital analog signal
  5. 5. H.264 / H.265 Encode PROCESS OF PLAYINGVIDEO 2. Encode video file Raw data .mp4 / .ts file Easier for storage and transference
  6. 6. PROCESS OF PLAYINGVIDEO 3. Play it ! .mp4 / .ts file a/v sync codec decode and display extract content from container
  7. 7. WAIT,WHERE IS PLAYER?
  8. 8. WHAT PLAYER DOES .mp4 / .ts file a/v sync codec decode and display HERE extract content from container
  9. 9. THEY ARE ALREADY HANDLED .mp4 / .ts file HTML5 a/v sync codec decode and display extract content from container
  10. 10. SO WHY DEVELOPING THIS PLAYER?
  11. 11. OUR PLAYER FOCUS ON (1) extract content and a/v info from container .mp4 / .ts file get files in smart way a/v sync codec decode and display
  12. 12. HOW PLAYER GET FILE ?
  13. 13. server holds .mp4 content VINTAGE WAY Player get it and play single file slow and inflexible
  14. 14. server holds .mp4 content EFFICIENT WAY Player get part of video and start play progressive download Keep requesting rest part of video during playback still inflexible
  15. 15. prepare different qualities of content on server MODERN WAY adaptive streaming Player plays best quality depends on network status Keep requesting rest part of video during playback
  16. 16. HOW ABOUT WE SPLIT FILE • split file into segments, described by manifest • manifest tells - available bitrates - how file is fragmented - other information, e.g encryption + +
  17. 17. knows how to get fragments ADAPTIVE STREAMING IN SHORT fragments manifest file
  18. 18. ADAPTIVE STREAMINGTYPES SS Smooth Streaming HLS HTTP Live Streaming HDS
  19. 19. INTRODUCE DASH Dynamic Adaptive Streaming through HTTP
  20. 20. PROS OF DASH • open source, with industry standard, which means more universal usage • support 2 types of containers, the MPEG-4 file format or the MPEG-2Transport Stream
  21. 21. GET FILES IN SMART WAY • manifest parser - able to read different type of manifest • adaptive bitrate - able to decide which quality to load
  22. 22. OUR PLAYER FOCUS ON (2) .mp4 / .ts file get files in smart way protection logic extract content from container a/v sync codec decode and display
  23. 23. DRM ?
 (DIGITAL RIGHTS MANAGEMENT) Content provider / DRM provider likes it For end user
  24. 24. CLIENT SIDE PROTECTION • Client get a license instead of a key from DRM service • A blackbox or sandbox get the key by processing license • That blackbox / sandbox decrypt content and directly output it to display
  25. 25. H.264 / H.265 Encode HOWTO PROTECT CONTENT Raw data .mp4 / .ts file DRM implemented here
  26. 26. PROTECTION LOGIC • give what DRM server needs and retrieve license • negotiate with browser in order to implement protection on client side • deal with different browsers knows how to
  27. 27. NOW, LET’S STARTTALK ABOUT BROWSER <video>, MSE and EME
  28. 28. HTML5VIDEO ELEMENT • <video></video> support progressive download • For adaptive streaming, MSE is required
  29. 29. extract content from container WHAT ROLE MSE PLAYS .mp4 / .ts file MSE handles a/v sync codec decode and display MSE handles
  30. 30. INTRODUCE MSE • Given different bitrate segments and it helps
 html5 video element to play smoothly • SourceBuffer provide video and audio buffer condition Media Source Extension
  31. 31. EME IS ABOUT PROTECTION
  32. 32. HOW BROWSER ADOPT DRM • For browser, that blackbox called CDM 
 (Content Decrypt Module) • Each browser support different DRM context - “a blackbox or sandbox get the key by processing license”
  33. 33. DRM ON BROWSER Widevine FairplayPlayready Primetime
  34. 34. INTRODUCE EME • Even though browser support its own DRM, 
 W3C defines a EME spec, in order to expose same api for client • prefixed api was implemented on earlier version of chrome(smart tv) Encrypted Media Extension
  35. 35. EME CDM provide context from encrypted content get ‘challenge’ DRM license server request with challenge get license provide license for CDM to decrypt content player EME CDM PROTECTION LOGIC FLOW
  36. 36. WHAT ROLE EME PLAYS extract content from container .mp4 / .ts file a/v sync codec decode and display decrypt content in blackbox
  37. 37. INTRODUCE OUR PLAYER
  38. 38. OUR GOAL • play not single file but sequence of segments, 
 with different bitrate, a.k.a adaptive streaming • play protected stuffs, for content providers’ goods a player able to
  39. 39. 影片 ya pi yapi.js
  40. 40. DEVELOP PROCESS make it work stable refactor
  41. 41. MAKE IT WORK FIRST only 2 files in the very beginning
  42. 42. BE STABLE • well structured • modularized • dependency management (dijon.js) • consistent code style
  43. 43. MODULARIZED api adaptive extension fragment manifest protection stats stream utils
  44. 44. UTILS • Capabilities • Debug • ErrorHandler • EventBus • IntervalBus • UrlModel api adaptive extension fragment manifest protection stats stream utils
  45. 45. EXTENSION • MediaSourceExtension • SourceBufferExtension • VideoModel api adaptive extension fragment manifest protection stats stream utils
  46. 46. MANIFEST • ManifestLoader • ManifestExtension • ManifestModel api adaptive extension fragment manifest protection stats stream utils
  47. 47. STREAM • StreamCtrl • BufferCtrl api adaptive extension fragment manifest protection stats stream utils
  48. 48. FRAGMENT • FragmentCtrl • FragmentLoader • SegmentTemplate api adaptive extension fragment manifest protection stats stream utils
  49. 49. ADAPTIVE • AbrCtrl • BandwidthRecorder api adaptive extension fragment manifest protection stats stream utils
  50. 50. PROTECTION • ProtectionCtrl • ProtectionModel • ProtectionRequest • Playready / Widevine api adaptive extension fragment manifest protection stats stream utils
  51. 51. STATS • MetricsCtrl • Metrics • StatsCtrl • BitrateHistory api adaptive extension fragment manifest protection stats stream utils
  52. 52. REFACTOR • review flow • redefine api
  53. 53. APPLICATION FLOW
  54. 54. REDEFINE API • use jsdoc to generate spec document • define api, event and vo(value object) • spec • result
  55. 55. DEMO Sample player
  56. 56. HOWTO MANAGE THOSE MODULES
  57. 57. DEPENDENCY INJECTION • not necessary in the beginning, but became very important after it went much more complex • use dijon.js as di framework
  58. 58. INTRODUCE DIJON.JS • a system holds all dependencies, after mapping modules to it • an object get dependencies after injection
  59. 59. DEFINE DEPENDENCIES // A dep function A() {} // B requires A function B() { this.a = undefined; // how dijon knows where to inject } // instantiate dijon var system = new dijon.System();
  60. 60. 3 KINDS OF MAPPING var a = new A(); system.mapValue(‘aValue’, a); // system.getObject(‘a’) would return an ‘a’ instance system.mapSingleton(aSingleton,A); // system.getObject(‘aSingleton’) would return a singleton a system.mapClass(bClass, B); // system.getObject(‘bClass’) would return a new inited B
  61. 61. OUTLET MAPPING // map outlet to make dependency work // @param sourceKey {String} - the key mapped to system would be injected // @param targetKey {String|'global'} - the key outlet is assigned to // @param outletName {String|sourceKey} - the name of property used as an outlet system.mapOutlet('aSingleton', 'bClass',‘a'); var b = system.getObject(‘bClass’); // b.a is aSingleton
  62. 62. DIRECTLY INJECTION var b = new B(); system.injectInto(b); // b WOULDN’T have dependency singleton A // b/c we only mapOutlet to key bClass of system // map outlet to global system.mapOutlet(‘aSingleton’,‘global’,‘a’); system.injectInto(b); // this would work
  63. 63. AUTO MAP OUTLET function B(){ this.aSingleton = undefined; this.c = undefined; } function C(){} // auto mapped outlet would assign to ‘global’ // and outlet name is the same as mapped key (before mapping) system.autoMapOutlets = true; system.mapClass(‘bClass’, B); // map again b/c B is changed system.mapClass(‘c’, C); // system.getObject(‘b’) would have c dep
  64. 64. COMMUNICATE IN BETWEEN Now B has dependencies of A and C How would you do in this scenario: A is doing something when it’s done, invoke a method of B –this is a very common situation in player
  65. 65. MAP HANDLER AND NOTIFY function A() { this.say = function() { system.notify(‘aDoneSaying’); }; } function B() { this.a = undefined; this.afterAsays = function(){ // do something }; } system.mapSingleton(‘a’,A); system.mapSingleton(‘b’, B); system.mapHandler(‘aDoneSaying’, ’b’,‘afterAsays’); // system.getObject(‘b’).a.say() would invoke b.afterAsays
  66. 66. NOTIFYING CLASS system.mapSingleton(‘a’,A); system.mapClass(‘b’, B); // map class here var b1 = system.getObject(‘b’); // b1.say() would invoke a newly instantiated b.afterAsays // instead of b1.afterAsays system.mapValue(‘b1’, b1); system.unmapHandler(‘aDoneSaying’,‘b’,‘afterAsays’) system.mapHandler(‘aDoneSaying’,‘b1’,‘afterAsays’) // b1.say() would invoke b1.afterAsays
  67. 67. function E() { this.a = undefined; this.setup = function () {}; } var e = new E(); system.injectInto(e); // e.setup invoked CONVENTION setup method of module would be invoked after getObject or injectInto
  68. 68. INJECT SYSTEM ITSELF system.mapValue('system', system); function D() { this.system = undefined; } system.mapSingleton('d', D); // system.getObject(‘d’) has system as dependency
  69. 69. function A() {}; function B() {} function Dep() { return { system: undefined, setup: function () { this.system.autoMapOutlets = true; // map dep here this.system.mapSingleton('a',A); this.system.mapClass('b', B); } }; } function App() { var b; var system = new dijon.System(); system.mapValue('system', system); system.mapOutlet('system'); var dep = new Dep(); system.injectInto(dep); // inject system to dep and invoke setup function, which map all deps return { system: undefined, a: undefined, setup: function () { b = system.getObject('b'); }, init: function () { system.injectInto(this); // after init, app.a exists } }; } // after new App().init(), app get all dep setup in Dep
  70. 70. SUMMARY OF DIJONJS • 3kb • very little convention • focus on di and notify / handler
  71. 71. BUILD PROJECT • list all source files in index.html of testing page • grunt as task manager • concat and minify scripts with grunt-usemin • export to dist/player.min.js
  72. 72. ONE LAST NOTICE • ui should be totally separated • ui <-> player -> videoElement • ui order player by api, respond to player behavior by event player videoElement ui
  73. 73. FUTURE OFYAPI.JS • firefox / safari drm support • live • ads • smart tv / chromecast / nexus player
  74. 74. THANKYOU questions ?

×