Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

November 2018 Seattle Video Tech Meetup: WAVE Update

116 views

Published on

Update on the Web Application Video Ecosystem (WAVE) Project by John Simmons of Microsoft.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

November 2018 Seattle Video Tech Meetup: WAVE Update

  1. 1. WAVE The Web Application Video Ecosystem project John Simmons Media Platform Architect Microsoft johnsim@microsoft.com
  2. 2. Web Application Video Ecosystem WAVE November 2018 WAVE Background • WAVE history and origin • WAVE relationship to global media standards WAVE Today • The what and who of WAVE • WAVE and content • WAVE and applications • WAVE and devices WAVE Future • WAVE and progressive web apps • The why of WAVE Discussion 2
  3. 3. WAVE History and Origin: 2008-2015 October 2008 Microsoft Athens Project to standardize DRM-interoperable Streaming September 2009 Publication of the PIFF container format and Smooth Streaming Transport March 2010 MPEG Call for Proposal to standardize HTTP Streaming February 2011 Microsoft and Netflix Proposals to W3C to support DRM and Adaptive Streaming February 2012 Dynamic Adaptive Streaming over HTTP (DASH) and Common Encryption published June 2012 HTML5 Encrypted Media Extensions (EME) and Media Source Extensions (MSE) Start April 2014 Microsoft “Interoperability, Digital Rights Management and the Web” paper delivered at NAB January 2015 Microsoft and Apple disclose CMAF project under NDA at CES. And the CTA proposes WAVE. December 2015 The Consumer Technology Association (CTA) launches WAVE project based on Microsoft NAB paper November 2018 3
  4. 4. WAVE History and Origin: 2016-present January 2016 First CTA Web Application Video Ecosystem (WAVE) meeting held at CES in Las Vegas February 2016 CMAF proposal submitted to MPEG supported by Adobe, Akamai, Apple, BBC, Cisco, Comcast, DTG, Ericsson, Fraunhofer, iStreamPlanet, LG, Microsoft, BAMTech, Stars, Telecom Italia, Turner and Verimatrix. April 2016 First WAVE Technical Working Group meeting held at NAB November 2016 HTML5 Media Source Extensions (MSE) a formal W3C Recommendation September 2017 HTML5 Encrypted Media Extensions (EME) a formal W3C Recommendation December 2017 CTA WAVE publishes “Web Application Video Ecosystem - Web Media API Snapshot 2017” January 2018 Common Media Application Format (CMAF) becomes a published ISO Specification April 2018 CTA WAVE publishes “Web Application Video Ecosystem – Content Specification” November 2018 4
  5. 5. WAVE AND GLOBAL WEB MEDIA STANDARDS ISO MPEG CENC ISO MPEG DASH Apple published in IETF November 2018 5
  6. 6. DRM 3 Proprietary Container DRM 3 Proprietary Encoding and Encryption BEFORE COMMON ENCRYPTION Common Container DRM1 Header DRM2 Header DRM3 Header Common Encoding and Encryption ISO MPEG CENC DRM 2 Proprietary Container DRM 2 Proprietary Encoding and Encryption DRM 1 Proprietary Container DRM 1 Proprietary Encoding and Encryption WITH COMMON ENCRYPTION November 2018 6
  7. 7. ▪ An 18-month Apple-Microsoft co-development activity. January 2015, Microsoft and Apple shared their idea of a common media format between HLS and DASH ▪ Proposed at MPEG’s 114th meeting in San Diego in Feb 2016. ▪ Requirements submitted by: Adobe, Akamai, Apple, BBC, Cisco, Comcast, DTG, Ericsson, Fraunhofer, iStreamPlanet, LG Electronics, Microsoft, MLBAM, Qualcomm, Samsung, Starz, Telecom Italia, Turner, Verimatrix, WWE. ▪ Draft specification submitted by: Apple, Microsoft, MLBAM, Cisco, Akamai and Comcast. ▪ Became a published ISO/IEC spec January 2018. CMAF was designed with live linear streaming, content protection and ad signaling in mind. November 2018 7
  8. 8. CDN delivery Cloud encoding mpd m3u8 CMAF mpd m3u8 CMAF mpd m3u8 CMAF mpd m3u8 CMAF mpd m3u8 CMAF mpd m3u8 CMAF • June 15, 2016 Apple announced HLS would support CMAF. • Now the top two adaptive delivery manifest formats - DASH and HLS - support the same content packaging 1) DASH, HLS with CMAF reduce storage and packaging costs, ISO MPEG DASH Apple published in IETF 2) improve edge caching efficiency - very important for live linear streaming, Origin Storage Source 3) and enable device interoperability. Mobile or PC App Smart TV App Game Machine App Set top box appNovember 2018 8
  9. 9. Player Framework HTML5 Media Application Web Application APIs EME MSE MCE Device Platform DRM Media Decoder Mobile or PC Game ConsoleSmart TV Set top box ▪ Media players use existing player frameworks – e.g. open source Shaka player, dash.js, hls.js and video.js, proprietary jwplayer, THEOplayer and the Bitmovin player. ▪ Encrypted Media Extensions (EME) interface to embedded DRM, Media Source Extensions (MSE) interface to the Media Decoder for adaptive playback. ▪ A Media Capabilities Extension (MCE) is being incubated which will enable device capability discovery. ▪ CTA WAVE and the W3C are co-publishing recommendations on embedded device browser API support.November 2018 9
  10. 10. Web Media Interop For Content, Application Platform and Devices – through specifications and industry engagement November 2018 10
  11. 11. WAVE Specification Work ▪ WAVE addresses global Web media interop issues by defining interop points based on global standards, targeting desktop and embedded browsers – laptops, phones, tables, smart TVs, media sticks and set-top boxes. ▪ Encoding issues are being worked on by the Content Specification Task Force, published in the WAVE content specification, based on profiles of the new ISO IEC CMAF specification. ▪ Platform issues are being addressed by the HTML5 API Task Force, in the published Web Media API Snapshot spec, as well as the upcoming Web Media Application Developers Guidelines and the Web Media Porting specification. ▪ Playback issues are being worked on by the Device Playback Capabilities Task Force, in the upcoming Device Playback Capabilities specification. ▪ WAVE Specs are free - https://cta.tech/WAVE November 2018 11
  12. 12. WAVE engagement with Industry ▪ WAVE addresses global media interop issues by working with Industry Fora (e.g. DASH IF, W3C) and SDOs (e.g. ISO MPEG) to drive improvements in baseline standards. ▪ Encoding improvements are being address by getting CMAF Media Profiles published in organizations like ETSI and the Alliance for Open Media, and by driving enhancements to the base CMAF specification, e.g. to address multi-stream audio formats and splice conditioning. ▪ Platform improvements are being addressed by promoting changes to HTML5 EME, MSE, Media Capabilities and Timed Data APIs (e.g. emsg box). ▪ Playback improvements are being addressed by developing test WAVE compliance testing for WAVE media profiles and the WAVE Web Media API Snapshot spec. ▪ Join WAVE by emailing standards@CTA.tech November 2018 12
  13. 13. WAVE Organization Steering Committee Technical Working Group Content Specification Task Force Device Playback Capabilities Task Force HTML5 API Task Force Will Law Akamai Mark Vickers Comcast Thomas Stockhammer Qualcomm John Simmons Microsoft Microsoft, Qualcomm, Samsung, Sky, Sony, Starz, TP Vision, WWE Adobe, Akamai, Apple, Comcast, Disney Streaming, Google, LG, November 2018 13
  14. 14. Adobe Systems, Inc AGP, LLC Akamai Amazon.com Apple AT&T Inc AwoX BBC Research & Development BitRouter Brightcove Inc. Cable Television Labs castLabs Comcast Cable Consumer Technology Association Cox Communications, Inc. Digital Entertainment Content Ecosystem (DECE) Discovery Communications Disney Streaming Disney/ABC/ESPN Dolby Laboratories Inc. DTS, Inc. Ericsson Eurofins Digital Testing Facebook Fraunhofer Google Inc. Home Box Office (HBO) Intel Corporation JW Player LG Electronics Microsoft Corporation Motion Picture Association of America Motion Picture Laboratories Nagravision National Association of Broadcasters Netflix, Inc. Nevelex P Thomsen Consulting Qualcomm Incorporated RK Entertainment Technology Consulting LLC Samsung Electronics SERAPHIC Information Technology SET - Brazilian Society of Television Engineering Sharp Electronics Corporation Sky Society of Cable Telecommunications Engineers Solekai Systems Sony Electronics SpireSpark International Starz Streamroot TBT, Inc. Toshiba TP Vision Holding B.V. Turner UStudio Verance Corporation Verimatrix Verizon Vewd Software Viacom Vizio WJR Consulting World Wide Web Consortium (W3C) WWE WAVE Membership November 2018 14
  15. 15. CMAF Codec bindings or “Media Profiles” with market support will be published annually in the CTA WAVE Content Specification. ... ... Some media profiles are identified in the ISO MPEG CMAF specification.Additional media profiles will be published elsewhere. CMAF Codec bindings to Global ISO MPEG and non-ISO MPEG Codecs, including new royalty free codecs like AV1 from the Alliance for Open Media. November 2018 15
  16. 16. WAVE Content Specification 2018 - Video Profiles Media Profile Name INFORMATIVE Codec INFORMATIVE Profile INFORMATIVE Level INFORMATIVE Color primaries & matrix coefficients INFORMATIVE Transfer Characteristics INFORMATIVE ‘codecs’ MIME subparameters NORMATIVE CMAF Brand NORMATIVE Normative Reference HD AVC High 4.0 1 (BT.709) 1 (BT.709 OETF) avc1.640028 avc3.640028 ‘cfhd’ [CMAF] Table A.1 HHD10 HEVC Main10 MainTier 4.1 1 (BT.709) 1 (BT.709) hev1.2.4.L123.B0 hvc1.2.4.L123.B0 ‘chh1’ [CMAF] Table B.1 UHD10 HEVC Main10 MainTier 10-bit 5.1 1 (BT.709) 9 (BT.2020) 1 (BT.709 OETF) 14 (BT.2020 OETF) hev1.2.4.L153.B0 hvc1.2.4.L153.B0 ‘cud1’ [CMAF] Table B.1 HLG10 HEVC Main10 MainTier 10-bit 5.1 9 (BT-2020) 18 (BT.2100 Table 5 HLG OETF) 14 (BT.2020 OETF) hev1.2.4.L153.B0 hvc1.2.4.L153.B0 ‘clg1’ [CMAF] Table B.1 HDR10 HEVC Main10 MainTier 10-bit 5.1 9 (BT.2020) 16 (BT.2100 Table 4 PQ EOTF) hev1.2.4.L153.B0 hvc1.2.4.L153.B0 ‘chd1’ [CMAF] Table B.1 The 2018 Edition of the WAVE Content Specification includes the following video Media Profiles. Additional media profiles are likely to be added in an amendment prior to the 2019 edition of the WAVE Content Specification – including a CMAF binding for AV-1.November 2018 16
  17. 17. WAVE Content Spec 2018 - Audio Profiles Media Profile Name INFORMATIVE Codec Family INFORMATIVE Allowed Codecs or Profiles INFORMATIVE Level INFORMATIVE ‘codecs’ MIME subparameter NORMATIVE CMAF Brand NORMATIVE Normative Reference AAC Core AAC AAC-LC, HE- AAC or HE- AAC v2 2 mp4a.40.2 mp4a.40.5 mp4a.40.29 ‘caac’ [CMAF] Table A.2 Adaptive AAC Core AAC AAC-LC, HE- AAC or HE- AAC v2 2 mp4a.40.2 mp4a.40.5 mp4a.40.29 ‘caaa’ [CMAF] Table A.2 AAC Multichannel AAC AAC-LC, HE- AAC 6 mp4a.40.2 mp4a.40.5 mp4a.40.29 ‘camc’ [CMAF A1] Table i.2 Enhanced AC- 3, including AC-3 AC-3 EAC-3 AC-3 EAC-3 n.a. ec-3 ‘ceac’ [EAC3] AC-4, Single Stream AC-4 AC-4 3 ac-4.02.01.03 ‘ca4s’ [AC4] MPEG-H, Single Stream MPEG-H Low Complexity (LC) 3 mhm1.0x0B mhm1.0x0C mhm1.0x0D ‘cmhs’ [CMAF A1] Table j.2 ▪ Some organizations outside MPEG have begun publishing bindings specifications for CMAF. ▪ The first of these organizations is ETSI, which is publishing CMAF bindings specifications for both Dolby and DTS audio codecs. ▪ Other organization have suggested they will publish CMAF bindings in 2018. ▪ The WAVE content specification also includes both IMSC1 Text and Image CMAF bindings. ▪ IMSC1 is being standardized because of its compatibility with EBU_TT_D used in Europe.November 2018 17
  18. 18. WAVE Programs and Live Linear Content WAVE Program CMAF Presentation 1 Audio Switching Set Video Switching Set Subtitle Switching Set CMAF Presentation 2 Audio Switching Set Video Switching Set CMAF Presentation 3 Audio Switching Set Video Switching Set Subtitle Switching Set Continuous Rendering for a continuous user experience ▪ Live linear content with ad insertions may require delivering not one but a sequence of CMAF Presentations. ▪ To address this likelihood, the WAVE content spec defines WAVE programs, which are a sequence of one or more CMAF Presentations. ▪ When a WAVE Program includes more than one CMAF presentations, it can optionally conform to a WAVE Splice Constraint Profile. ▪ The 2018 Edition of the WAVE Content Spec defines a Baseline Splice Constraint Profile, encoding constraints intended to allow continuous rendering of Sequential Switching Sets in WAVE Programs on most existing adaptive streaming Players in the market today. ▪ We anticipate that as new devices enter the market, the more advanced Splice Constraint Profiles will be published by WAVE. November 2018 18
  19. 19. HTML5 APIs: Reference Platform 19 WAVE/CMAF One Content Format... Reference Platform HTML5 tests Write reference tests in HTML5… … then port tests to device platforms. (HTML5 platforms run tests directly.) ...but multiple devices mobile apps PC apps TV apps game apps set-top apps
  20. 20. HTM5 API Task Force: Work Plan • Web Media API Community Group: • w3.org/community/webmediaapi/ 1. Annual Web Media API spec • define baseline web APIs to support media web apps. 2. Guidelines for media web app developers 3. Identify gaps in current web APIs • work with W3C Working Groups to update web standards. 20
  21. 21. Web Media API Snapshot 2017 • First annual API Snapshot published 20 December 2017: https://www.w3.org/2017/12/webmediaapi.html ▪ Lists key APIs supported in 2017 in all major HTML code bases. ▪ CTA-W3C agreement to co-publish this spec. ▪ Plan to propose Community Group spec as a W3C standards track spec ▪ CTA WAVE released a test suite for all listed APIs based on W3C API tests (https://webapitests2017.ctawave.org). ▪ Test suite will enable manufacturers to test that their HTML support is up-to-date! 21
  22. 22. OTT Device Performance Challenges • Ad splicing problems • Regional profiles (50/60Hz) • Request protocol deficiencies • Unknown codec capabilities • Unknown rendering capabilities • Partial profile support • Codec incompatibility • Long-term playback instability • Late Binding Synchronization • Audio discontinuities • Glitches when switching bitrate • Memory problems • Limited processing power • Long start-up delay • Performance monitoring • DRM support • Variable HDR support • Scaling display issues 22
  23. 23. Device Types • WAVE includes different types of OTT clients, both HTML and Native • Requirements apply universally, tests focused on (referenced to) HTML 23 App-driven players: Native App Devices Native Platform Player HTML5 App Player JavaScript & MSE HTML5 Platform Player video object Native App Player Embedded players: HTML5 App Devices
  24. 24. Video Source Buffer Audio Source Buffer Abstracted Device Playback Model 24 Device Playback Model Video Source Buffer Audio Source Buffer Subtitle Source Buffer Visual Rendering Audio Rendering Device Capabilities Stimulus/Input Observations Requirements: If you input WAVE content, this shall be the observation
  25. 25. DPCTF Specification Objectives • Provide testable requirements for device performance challenges • Provide capability code points for WAVE content • Enable the qualification of existing platforms for their WAVE content playback capabilities • Generate a forward-looking specification for advanced media playback requirements, including new codecs and experiences • Prioritize challenges and address the highest priority items first 25
  26. 26. November 2018 26
  27. 27. Mobile or PC Game ConsoleSmart TV Set top box EdgeHTML Gecko WebKitBlink APP-LIKE Feels like an app, works like a website - Progressive Web Apps (PWAs) are powered by Web technology and delivered with Web Infrastructure. PROGRESSIVE PWAs will be supported by most embedded and desktop web browser engines. RESPONSIVE Easily built targeting multiple form factor devices, using the new W3C Manifest for Web Apps spec. BUT NETWORK INDEPENDENT Works with or without network connection for offline playback, thanks to "service workers" BUT ALWAYS FRESH PWAs are always up to date, updates do not need to be resubmitted to store. DISCOVERABLE AND RE- ENGAGEABLE Search engines can find the app, because of scope of W3C manifest for Web Apps and Web Worker registration – and supports push notifications. INSTALLABLE AND LINKABLE Installs like a native application and can be shared with a URL – e.g. from the base website. November 2018 27
  28. 28. November 2018 28 WHY WAVE?
  29. 29. Internet Inflection Point in 1989 November 2018 29
  30. 30. Web Inflection Point in 1993 November 2018 30
  31. 31. Hypertext Transport Protocol (HTTP) Hypertext Markup Language (HTML) Portable Browser (Mosaic) Mobility of Internet Experience (Web) Common Media Application Format (CMAF) Dynamic Adaptive Streaming over HTTP (DASH) HTML Media Extensions (EME, MSE) Progressive Web Apps (PWAs) Mobility of Web Media Experience 2018 1993 Global Web Media Standards Inflection Point November 2018 31
  32. 32. Three Tiers of Commercial Web Media Tier-1 mainstream: linear content from cable/satellite/telco, most popular and limited in variety because linear is bandwidth constrained Tier-2 direct-to-consumer: Tier-1 channels or channel programs going direct to consumer. Inherit interop challenge. Tier-3 Long tail: vertical niche content made practical by global Internet standards for commercial media which are cloud addressable objects. “Farm league” for linear content. MassMarket Niche Market November 2018 32
  33. 33. Devices Mobile, Game Console, Smart TV Mobile, Game Console, Smart TV Impact of Global Industry Adoption Set top box Desktop Set top box Content Industry Tier-1 Mainstream Tier-2 Direct to Consumer Tier-3 Long Tail AuthZ Providers Cable/Sat/Telco Online Video Distributors Specialty providers Desktop Legacy Transition Future Set top box November 2018 33
  34. 34. ISO MPEG CENC ISO MPEG DASH Apple published in IETF November 2018 34
  35. 35. ISO MPEG CENC ISO MPEG DASH Apple published in IETF Web App Manifest, W3C Working Draft 08 May 2018, www.w3.org/TR/appmanifest/ Media Source Extensions, W3C Recommendation 17 November 2016, http://www.w3.org/TR/media-source/ Encrypted Media Extensions, W3C Recommendation 18 September 2017, http://www.w3.org/TR/encrypted-media/ HTTP Live Streaming (HLS), RFC8216, https://tools.ietf.org/html/rfc8216 ISO/IEC 23009-1:2014, Information technology – Dynamic adaptive streaming over HTTP (DASH) – Part 1: Media presentation description and segment formats, https://www.iso.org/standard/65274.html ISO/IEC 23000-19, Information technology — Coding of audio-visual objects — Part 19: Common media application format (CMAF) for segmented media. https://www.iso.org/standard/71975.html ISO/IEC 23001-7:2016, Information technology – MPEG systems technologies – Part 7: Common encryption in ISO base media file format files, https://www.iso.org/standard/68042.html November 2018 35
  36. 36. Discussion November 2018 36

×