WebRTC: Google Notes
KrankyGeek, Sep 2015
•
Our 40 minutes together
❖ Vital Signs
❖ WebRTC News
❖ Public Service Announcements
❖ Recent Improvements
❖ Coming Attracti...
Some vital signs...
•
Vital Signs: WebRTC Adoption
❖ Tracking 720 companies using WebRTC in some form
➢ Up from 550 this past February
❖ Many ma...
❖ 28 WebRTC related acquisitions to date
❖ Some recent ones:
➢ Acision
➢ Speek
➢ mPortal
➢ Tropo
➢ ESNA
➢ Screenhero
➢ Jit...
And now, the news.
•
News: An open alliance for media (1/2)
Focused on developing next generation media formats,
codecs and technologies in the...
News: An open alliance for media (2/2)
The Alliance’s initial focus is to deliver a next-generation video format that is:
...
News: Codecs, Codecs, Codecs
❖ Many companies actively contributing to open codecs:
Google (VP9), Mozilla (Daala), Cisco (...
News: Microsoft Edge
❖ A brand new web browser, with support for
getUserMedia and ORTC (coming soon).
❖ Reported to includ...
Public Service Announcements
PSA: IP Address 'Leakage'
Recap
❖ WebRTC is a P2P stack
❖ P2P means trying to find the fastest route. A fastest route may
...
PSA: IP Address 'Leakage'
TL;DR
❖ The behavior of the stack was by design
❖ However, the behavior of users has changed
❖ W...
Network Limiter Extension
Network Limiter Extension
❖ Step 1: Network Limiter Extension
➢ Makes WebRTC traffic follow default route -
even if this i...
Next Steps
❖ Step 2: Make Default
➢ Once issues are resolved (a few remain),
turn this behavior on by default
➢ Sites with...
PSA: Mandatory HTTPS
Chrome 47
rolling out in December 2015
WILL NOT ALLOW
getUserMedia requests
from http://
•
PSA: HTTPS
One. more. time:
NO getUserMedia from HTTP
starting in December
trusted origins only!
•
PSA: ECDSA
❖ Elliptic Curve Digital Signature Algorithm (ECDSA)
➢ Reduces call setup crypto overhead from 1000 -> 50ms
➢ M...
PSA: DTLS 1.2
❖ DTLS 1.2
➢ Rolling out in Chrome 46-47 timeframe
➢ Should be mostly seamless; falls back to 1.0
➢ Allows m...
PSA: test.webrtc.org
Open source tool that tests:
❖ Network conditions (bw, firewalls, etc…)
❖ If microphone is connected ...
PSA: Report it!
Simple Serge / Justin rule:
"If you did not report it, it did not happen."
http://www.webrtc.org/report-bu...
Enhancements
Enhancements: Delay Agnostic - AEC
❖ 99% on for Windows
❖ 99% on for Linux
❖ 99% on for ChromeOS
❖ Mac next week,
after en...
Enhancements: AEC rollout troubles
Rise in full echo reports in June
❖ Caused by condition where interface between the ren...
Enhancements: Screensharing
❖ Faster screen sharing in Chrome 45
❖ Improved handling of complex scene switches
❖ Better su...
Enhancements: Screensharing
•
Enhancements: Mobile performance
❖ 5x faster rendering code
➢ Encoding and rendering local video from textures
❖ Capture p...
Enhancements: Mobile power savings
•
Enhancements: Mobile audio
❖ New Android audio stack
➢ 40% reduction in loopback latency on devices that support Open SL
❖...
Enhancements: Mobile video
❖ VP8 + H.264 HW accel on Android
➢ For Android 5.0+
➢ Works best on Qualcomm chipsets
❖ H.264 ...
Enhancements: Video smoothness
Chrome 45
Enhancements: Video smoothness
Chrome 46
Enhancements: Chrome on desktop
❖ New JS API for selecting audio output devices
➢ HTMLMediaElement.setSinkId()
➢ Currently...
Around the corner
Around the corner: iOS API revamp
❖ v2 API underway
➢ Adjusts API to mimic JS API more closely
➢ Many cosmetic changes to ...
Around the corner: VP9
❖ On (no flag) in M47, with standard packetization
❖ Initial rollout will be single layer
➢ "40% fe...
Around the corner: H.264 in Chrome
❖ Working in limited scenarios on mobile and desktop
❖ Initial Chrome release will be s...
Around the corner: MediaStreamRecorder
Top request on chromium.org (over 2500 stars)
❖ Initial focus on recording local Me...
Around the corner: Misc
❖ Faster ICE, TURN setup
❖ Instant Wifi/Cell handoff
❖ More efficient radio management
Icebox
❖ Remote MediaStream processing in Web Audio
➢ crbug.com/121673
❖ Unified Plan
➢ crbug.com/465349
❖ WebRTC in Chrom...
WebRTC/ORTC (1/2)
❖ WebRTC 1.0 targeting Last Call* by EOY2015
➢ Incorporates many ORTC objects
(RtpSender/Receiver, Ice/D...
WebRTC/ORTC (2/2)
❖ After WebRTC 1.0? WebRTC NV
❖ WebRTC NV fully converges ORTC with WebRTC 1.0
❖ Apps can program to 1.0...
WebRTC/ORTC convergent,
not divergent
(Yay!)
That’s all folks
Upcoming SlideShare
Loading in …5
×

Kranky Geek WebRTC 2015 - What's next for WebRTC?

1,300 views

Published on

lways the highlight of this event, Google provides the nitty gritty details on what they are doing to progress WebRTC and their internal developments to help your application succeed. We'll cover more details on the recently announced Alliance for Open Media, an effort to create royalty-free video codecs.

Google Team

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,300
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
40
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Kranky Geek WebRTC 2015 - What's next for WebRTC?

  1. 1. WebRTC: Google Notes KrankyGeek, Sep 2015
  2. 2.
  3. 3. Our 40 minutes together ❖ Vital Signs ❖ WebRTC News ❖ Public Service Announcements ❖ Recent Improvements ❖ Coming Attractions •
  4. 4. Some vital signs... •
  5. 5. Vital Signs: WebRTC Adoption ❖ Tracking 720 companies using WebRTC in some form ➢ Up from 550 this past February ❖ Many major applications using WebRTC, confirmed by analysis on webrtchacks.com ➢ Hangouts ➢ Facebook Messenger ➢ Mayday ➢ Wire •
  6. 6. ❖ 28 WebRTC related acquisitions to date ❖ Some recent ones: ➢ Acision ➢ Speek ➢ mPortal ➢ Tropo ➢ ESNA ➢ Screenhero ➢ Jitsi Vital Signs: WebRTC Adoption •
  7. 7. And now, the news. •
  8. 8. News: An open alliance for media (1/2) Focused on developing next generation media formats, codecs and technologies in the public interest. Founding members: Amazon, Netflix, Microsoft, Intel, Cisco, Mozilla, Google http://aomedia.org
  9. 9. News: An open alliance for media (2/2) The Alliance’s initial focus is to deliver a next-generation video format that is: ❖ Interoperable and open; ❖ Optimized for the web; ❖ Scalable to any modern device at any bandwidth; ❖ Designed with a low computational footprint and optimized for hardware; ❖ Capable of consistent, highest-quality, real-time video delivery; and ❖ Flexible for both commercial and non-commercial content, including user-generated content. Among the Alliance’s goals is the creation of a new, high-quality open video format that improves core media experiences for all.
  10. 10. News: Codecs, Codecs, Codecs ❖ Many companies actively contributing to open codecs: Google (VP9), Mozilla (Daala), Cisco (Thor) ❖ Microsoft announces VP9 support in their new Edge browser ❖ Apple apparently is dropping HEVC from iPhone 6s (source: 9 to 5 Mac)
  11. 11. News: Microsoft Edge ❖ A brand new web browser, with support for getUserMedia and ORTC (coming soon). ❖ Reported to include VP9 ❖ More on ORTC later :-)
  12. 12. Public Service Announcements
  13. 13. PSA: IP Address 'Leakage' Recap ❖ WebRTC is a P2P stack ❖ P2P means trying to find the fastest route. A fastest route may be locally on your LAN, or over your VPN ❖ Web pages may use data channels silently for CDN purposes ❖ This presents problems when users use VPN to hide their public IP addresses on the Internet
  14. 14. PSA: IP Address 'Leakage' TL;DR ❖ The behavior of the stack was by design ❖ However, the behavior of users has changed ❖ We believe we can solve this problem in a smart, safe way
  15. 15. Network Limiter Extension
  16. 16. Network Limiter Extension ❖ Step 1: Network Limiter Extension ➢ Makes WebRTC traffic follow default route - even if this isn't the 'optimal' path ➢ Explicit opt-in lets us safely understand what side effects it might have
  17. 17. Next Steps ❖ Step 2: Make Default ➢ Once issues are resolved (a few remain), turn this behavior on by default ➢ Sites with camera permission can still get previous behavior ➢ May result in subtle behavior changes for data channel apps (e.g. more relay) ➢ Try the extension with your app
  18. 18. PSA: Mandatory HTTPS Chrome 47 rolling out in December 2015 WILL NOT ALLOW getUserMedia requests from http:// •
  19. 19. PSA: HTTPS One. more. time: NO getUserMedia from HTTP starting in December trusted origins only! •
  20. 20. PSA: ECDSA ❖ Elliptic Curve Digital Signature Algorithm (ECDSA) ➢ Reduces call setup crypto overhead from 1000 -> 50ms ➢ More secure than RSA-1024 ❖ Step 1: Opt-in via API (Chrome 47) ➢ pc.generateCertificate({name: "ECDSA", namedCurve: "P- 256"}) ❖ Step 2: Default (Chrome 48 - 49) ➢ Test your system now
  21. 21. PSA: DTLS 1.2 ❖ DTLS 1.2 ➢ Rolling out in Chrome 46-47 timeframe ➢ Should be mostly seamless; falls back to 1.0 ➢ Allows more hardware offload of encryption ➢ Try it with your app: --enable-webrtc-dtls12
  22. 22. PSA: test.webrtc.org Open source tool that tests: ❖ Network conditions (bw, firewalls, etc…) ❖ If microphone is connected and working properly ❖ If camera is connected and working properly Please contribute: github.com/webrtc/testrtc
  23. 23. PSA: Report it! Simple Serge / Justin rule: "If you did not report it, it did not happen." http://www.webrtc.org/report-bug •
  24. 24. Enhancements
  25. 25. Enhancements: Delay Agnostic - AEC ❖ 99% on for Windows ❖ 99% on for Linux ❖ 99% on for ChromeOS ❖ Mac next week, after encouraging 1% experiment ❖ App control via googDAEchoCancellation constraint •
  26. 26. Enhancements: AEC rollout troubles Rise in full echo reports in June ❖ Caused by condition where interface between the render and browser processes causes audio duplication ❖ Throws old AEC off (which means Mac right now) ❖ New DA-AEC adapts after 1-2 seconds ❖ Still investigating why duplicates happen ❖ Apps can control •
  27. 27. Enhancements: Screensharing ❖ Faster screen sharing in Chrome 45 ❖ Improved handling of complex scene switches ❖ Better support for scrolling and spreadsheet borders ❖ More improvements waiting in 46 around BWE •
  28. 28. Enhancements: Screensharing •
  29. 29. Enhancements: Mobile performance ❖ 5x faster rendering code ➢ Encoding and rendering local video from textures ❖ Capture performance improvements ➢ No more frame copies. No more conversions. ❖ Battery and CPU usage ➢ 75% reduction in GPU usage ➢ ~50% reduction power consumption •
  30. 30. Enhancements: Mobile power savings •
  31. 31. Enhancements: Mobile audio ❖ New Android audio stack ➢ 40% reduction in loopback latency on devices that support Open SL ❖ Major improvements to iOS audio ➢ Early tests show round-trip latency reduced from ~90ms to ~60ms ➢ Complexity (CPU) has also been reduced (no figures yet). ➢ Many existing crash issues have been resolved •
  32. 32. Enhancements: Mobile video ❖ VP8 + H.264 HW accel on Android ➢ For Android 5.0+ ➢ Works best on Qualcomm chipsets ❖ H.264 HW accel on iOS ➢ For iOS 8+ ❖ Works well; further optimizations coming ❖ Track it: bugs.webrtc.org, issue 4081
  33. 33. Enhancements: Video smoothness Chrome 45
  34. 34. Enhancements: Video smoothness Chrome 46
  35. 35. Enhancements: Chrome on desktop ❖ New JS API for selecting audio output devices ➢ HTMLMediaElement.setSinkId() ➢ Currently behind flag; track it: crbug.com/438023 ❖ Updated API for device enumeration ➢ MediaDevices.enumerateDevices() ❖ Event-driven datachannel buffering ➢ DataChannel.onbufferedamountlow ❖ IPv6
  36. 36. Around the corner
  37. 37. Around the corner: iOS API revamp ❖ v2 API underway ➢ Adjusts API to mimic JS API more closely ➢ Many cosmetic changes to conform to ObjC guidelines ➢ Callbacks will come on main event queue ➢ Release via CocoaPods ❖ v2 API will be side-by-side with current API ➢ For some finite transition period...
  38. 38. Around the corner: VP9 ❖ On (no flag) in M47, with standard packetization ❖ Initial rollout will be single layer ➢ "40% fewer bits at 15% more CPU" ❖ Working with Vidyo on SVC support ➢ Temporal + Spatial with low overhead ➢ Still need to figure out APIs to control this ➢ Initial SVC rollout using flag
  39. 39. Around the corner: H.264 in Chrome ❖ Working in limited scenarios on mobile and desktop ❖ Initial Chrome release will be software codec* ❖ Will interop with WebRTC standalone on Android/iOS ❖ M48 is a good guess (beta by EOY 2015) ❖ Track it: crbug.com/500605
  40. 40. Around the corner: MediaStreamRecorder Top request on chromium.org (over 2500 stars) ❖ Initial focus on recording local MediaStreams ❖ Goal is to have it working behind a flag in Chrome 47 ➢ Your help in testing will be crucial ➢ Would love to hear about your use cases ❖ Track it: crbug.com/262211
  41. 41. Around the corner: Misc ❖ Faster ICE, TURN setup ❖ Instant Wifi/Cell handoff ❖ More efficient radio management
  42. 42. Icebox ❖ Remote MediaStream processing in Web Audio ➢ crbug.com/121673 ❖ Unified Plan ➢ crbug.com/465349 ❖ WebRTC in Chrome for iOS ➢ crbug.com/416226
  43. 43. WebRTC/ORTC (1/2) ❖ WebRTC 1.0 targeting Last Call* by EOY2015 ➢ Incorporates many ORTC objects (RtpSender/Receiver, Ice/Dtls/SctpTransport) ❖ New objects provide more direct control: ➢ Switch cameras/tracks/codecs on the fly ➢ Set maximum bitrate for an encoding ➢ Configure simulcast (hopefully) ❖ Coming to Chrome in 2016 (Maybe a little in 2015)
  44. 44. WebRTC/ORTC (2/2) ❖ After WebRTC 1.0? WebRTC NV ❖ WebRTC NV fully converges ORTC with WebRTC 1.0 ❖ Apps can program to 1.0 (high-level) or object APIs ➢ Complete bypass of SDP, if desired
  45. 45. WebRTC/ORTC convergent, not divergent (Yay!)
  46. 46. That’s all folks

×