This document discusses WebRTC and its capabilities and components. WebRTC allows for real-time communication like low-latency video calling directly in the browser. It includes APIs for accessing media devices, recording media, and setting up peer-to-peer connections. WebRTC works across browsers but support varies, and signaling, NAT traversal, or media servers may be needed for production apps. New versions of WebRTC will support newer codecs, transports, and allow for more advanced media processing and machine learning.
2. cwh.consulting
A blog for WebRTC developers
webrtcHacks.com
@webrtcHacks
AI & RTC blog
cogint.ai
@cogintai
WebRTC and ML for Developer Event
November 16, 2018 in San Francisco
krankygeek.com
About Me
Chad Hart
Analyst & Product Consultant
https://cwh.consulting
@chadwallacehart
chad@cwh.consulting
9. cwh.consulting
What you get with WebRTC:
Camera and microphone capture
navigator.mediaDevices.getUserMedia()
https://www.w3.org/TR/mediacapture-streams/#mediadevices
10. cwh.consulting
What you get with WebRTC:
List available media devices
navigator.mediaDevices.enumerateDevices()
https://www.w3.org/TR/mediacapture-streams/#dom-mediadevices-enumeratedevices
11. cwh.consulting
What you get with WebRTC:
Audio output device selection
HTMLMediaElement.setSinkId()
https://www.w3.org/TR/audio-output/
12. cwh.consulting
What you get with WebRTC:
Screensharing & desktop capture
navigator.mediaDevices.getDisplayMedia()
https://www.w3.org/TR/mediacapture-streams/#mediadevices
13. cwh.consulting
What you get with WebRTC:
Screensharing & desktop capture
navigator.mediaDevices.getDisplayMedia
h"ps://w3c.github.io/mediacapture-screen-share/
14. cwh.consulting
What you get with WebRTC:
Media recording
MediaRecorder
https://www.w3.org/TR/mediastream-recording/
15. cwh.consulting
What you get with WebRTC:
Peer-to-Peer Media
RTCPeerConnection
https://www.w3.org/TR/webrtc/#rtp-media-api
16. cwh.consulting
What you get with WebRTC:
Peer-to-Peer Data
RTCPeerConnection.createDataChannel()
https://www.w3.org/TR/webrtc/#peer-to-peer-data-api
0101 1010
17. cwh.consulting
What you get with WebRTC:
Connection statistics
RTCPeerConnection.getStats()
https://www.w3.org/TR/webrtc/#sec.stats-model
18. cwh.consulting
Vendor Google Microsoft Mozilla Microsoft Apple
Browser Chrome Internet Explorer Firefox Edge Safari
Desktop ! ⛔ ! !
Android ! # ! # #
iOS # #
!
⛔ !⛔
WebRTCBrowser Support
20. cwh.consulting
Types of WebRTC Servers
Signaling
NAT
Traversal
Media Gateway
Always needed Needed for production Depends on the app Depends on the app
21. cwh.consulting
Swatting NATs
STUN TURN
Traversal Using Relay NATSession Traversal U/li/es for NAT
Lightweight mechanism for finding a
device’s own public IP address behind a
NAT
Heavy mechanism that relays media
through a publicly addressed server
when other connectivity fails
22. cwh.consulting
Reasons you might need a media server
Mul$-party Recording
Heavy media
manipulation
Live
broadcast
Kranky Geek São Paulo 2016
23. cwh.consulting
The Mesh Problem
Client Client
Client
Client
Client
Client
Clients Streams
/ client
Total
6 10 30
You can only connect so many points
in a peer-to-peer mesh architecture
before it overwhelms local resources
24. cwh.consulting
Selective Forwarding
Unit (SFU)
Client Client
Client
Client
Client
Client
Clients Streams
/ client
Total
n n n2
SFU
A SFU is a kind of media servers.
Relaying media via a SFU is the
best-practice way to handle multi-
party conference calls with more
than a few participants.
27. cwh.consulting
WebRTC Next Version – more options
App
peerConnection
OG WebRTC WebRTC NV
Browser
Application
code
Hear more from Google’s Kranky Geek 2018 talk: hAps://youtu.be/gV354kWh08o?t=1232
Lower-level API allow more control of
WebRTC parameters and interactions
Possibility to put low-level processing like
codecs in Web Assembly
29. cwh.consulting
A blog for WebRTC developers
webrtcHacks.com
@webrtcHacks
AI & RTC blog
cogint.ai
@cogintai
WebRTC and ML for Developer Event
November 16, 2018 in San Francisco
krankygeek.com
About Me
Chad Hart
Analyst & Product Consultant
https://cwh.consulting
@chadwallacehart
chad@cwh.consulting