WebRTC
A front-end perspective
Opera Software
Fronteers
2014
@shwetank

About me
Shwetank Dixit
Opera Software
Fronteers
2014
@shwetank
Developer Relations, Opera Software

http://www.thehindu.com/news/national/other-states/using-pcos-in-mumbai-show-id-proof/article4241422.ece

Source: https://www.flickr.com/photos/rambow/858719954

Source: https://www.flickr.com/photos/rambow/858717314 Source: https://www.flickr.com/photos/rambow/858719954

What is
WebRTC?
Real Time
Peer-to-Peer
Communication

but …
So far I’ve seen very
few web people
talking about it

Whats the biggest barrier to WebRTC adoption?*
Lack of Awareness
Not supported by MS
Not supported by Apple
Other
0% 5.5% 11% 16.5% 22%
* WebRTC World Outlook Survey 2014

WebRTC is for
but … Web Developers

Mediastream

Mediastream
RTCPeerConnection

Mediastream
RTCPeerConnection
RTCDataChannel

mediaStream
+
` `
Device Camera Microphone

getUserMedia
+
` `
Device Camera Microphone

Sqwiggle

gUM + CSS(Filters, Masks, Blend Modes)
Create interesting visual effects from
camera output in real-time using CSS

gUM + Web Audio
Sound powered actions
and navigation

World’s first webcam
http://en.wikipedia.org/wiki/Trojan_Room_coffee_pot

What can you achieve with gUM?
- Motion Detection (Baby Monitors, Security Cam)
- Hotspots
- Gesture Recognition
- Emotion Recognition
- Sound Detection and Visualisation
- Voice / Sound Activated Controls
- Capture video (Kind of) (whammy.js, gifshot)

gUM: Current way to do it
navigator.getUserMedia*
*with prefixes

gUM: Broken Error Handling
Exceptions are not caught by the current API.
Promises can fix this.
Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Sep/0171.html

gUM: Current Way
var failure = function(reason){
log("Failed to show camera: " + reason.message);
}
!
navigator.getUserMedia(constraints, function(stream) {
try {
$video.src = window.URL.createObjectURL(stream);
$video.play();
} catch(e) { failure(e); } //failure must tolerate non-MediaStreamError
}, failure);
Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Sep/0171.html

gUM: In the future ?
navigator.mediaDevices.getUserMedia

gUM: In the future ? Promises Proposal
navigator.mediaDevices.getUserMedia(constraints)
.then( function(stream) {
videoElement.src = window.URL.createObjectURL(stream);
videoElement.play();
}).catch(function (reason){
log(“Failed to show camera: ” +reason.message);
});
Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Sep/0171.html

Debate
This is still up for debate.
Some are eager to just ship v1.0 and add support for
promises in gUM at a later stage.
!
Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Oct/0003.html

gUM: Other concerns
Impossible for a gUM app to
use the device flash for low-light
!
Use (auto) zoom
Use (auto) focus

gUM: Other concerns
“If you look further into this API then it
doesn’t offer any advanced features
that you expect in a good camera app:
focus, flash, zoom `
etc. It is clear that
…(getUserMedia).. has been designed
for use in P2P applications and not as a
dedicated camera experience.”
- Paul Kinlan

Mediastream Image-Capture
!
Will provide a deeper level of control
for taking photos
!
w3.org/TR/image-capture/

Mediastream Image-Capture
White Balance
ISO
Red-Eye Reduction
Exposure
Brightness
Contrast
`
Saturation
Sharpness
Zoom
takePhoto()

Mediastream Image-Capture
Focus? Flash? ¯_(ツ)_/¯

Getting permission is not easy
Some users don’t seem to
notice the permissions dialog
in certain browsers

Getting permission is not easy
Some users don’t seem to
notice the permissions dialog
in certain browsers

Some users don’t seem to
notice the permissions dialog
in certain browsers

Some users don’t seem to
notice the permissions dialog
in certain browsers

Some users don’t seem to
notice the permissions dialog
in certain browsers

Getting permission is not easy
Permissions dialogs are different on
different platforms and browsers …
`
especially mobile

HTTPS helps …
If the app is on HTTPS, then the
browser will remember the last
granted permission.

RTCPeerConnection

webrtcglossary.com

RTCPeerConnection

ICE - Interactive Connectivity Establishment
* May or may not wax a chump like a candle.

STUN
Whats my
external IP Address?
Your external
IP Address is x.x.x.x

TURN around…
Total eclipse of the (p2p) call …

TURN
Basically relays stuff from one
party to the other.

Big Fat TURN Server

TURN
Basically relays stuff from one
party to the other.
In this case, connection won’t
be p2p, but at least the
connection will be made!

TURN
Basically relays stuff from one
party to the other.
In this case, connection won’t
be p2p, but at least the
connection will be made!
Generally speaking, a TURN server will
include a STUN server too.

Free ICE Project
github.com/DamonOehlman/freeice
(List of free STUN/TURN servers to use in your projects)

The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon

Session Description (SDP)
v=0
o=- 680121471469462884 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio video
a=msid-semantic: WMS GUKF430Khp9jEQiPrdYe0LbTAALiNAKAIfl2
Audio Lines
m=audio 54278 RTP/SAVPF 111 103 104 0 8 106 105 13 126
c=IN IP4 180.6.6.6
a=rtcp:54278 IN IP4 180.6.6.6
!
ICE Candidates
a=candidate:4022866446 1 udp 2113937151 192.168.0.197 36768 typ host generation 0
a=candidate:4022866446 2 udp 2113937151 192.168.0.197 36768 typ host generation 0
a=candidate:2706108158 1 tcp 1509957375 192.168.0.197 0 typ host generation 0
a=candidate:2706108158 2 tcp 1509957375 192.168.0.197 0 typ host generation 0
a=candidate:1853887674 1 udp 1845501695 46.2.2.2 36768 typ srflx raddr 192.168.0.197 rport 36768 generation 0
a=candidate:1853887674 2 udp 1845501695 46.2.2.2 36768 typ srflx raddr 192.168.0.197 rport 36768 generation 0
a=candidate:2157334355 1 udp 33562367 180.6.6.6 54278 typ relay raddr 46.2.2.2 rport 38135 generation 0
a=candidate:2157334355 2 udp 33562367 180.6.6.6 54278 typ relay raddr 46.2.2.2 rport 38135 generation 0

The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Creates Offer
local description

The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Receives Offer
local description

The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Receives Offer
remote description
local description

The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Creates and
sends Answer
local description
local description
remote description

The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Receives Answer
local description
local description
remote description

The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Receives Answer
local description
local description
remote description
remote description

The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Connectivity
Checks
local description
local description
remote description
remote description

The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
P2P Connection Established

Libraries

Bandwidth Numbers
!
720p : 1-2 Mbps // VP8 at 30fps
360p : 0.5 - 1 Mbps // VP8 at 30fps
180p : 0.1 - 0.5 Mbps // VP8 at 30fps
Audio : .06 - 0.51 Mbps // Opus

Debugging
opera://webrtc-internals
chrome://webrtc-internals
about:webrtc
!
The getStats() method
!
callstats.io
!

Data Channels
Once RTCPeerConnection is
established, we can use Data Channels
to transfer data P2P.
Data can be text or binary data

Whats the advantage?
Data Channels provide a
* High Performance
* P2P
* Low Latency
Connection to others

Working with Data
datachannel
.onopen // Data Channel opened
.onclose // Data Channel closed
.onerror // Error occurred
.onmessage // New message arrived
.send() // Send a message

DataChannels can work
exactly like WebSockets if
you want, but they don’t
have to …

SCTP // Transport Protocol
DTLS // Encryption
UDP

notable options
ordered
send data packets in ordered fashion or not
!
maxRetransmitTime
Max time to try to send a failed message
!
maxRetransmits
Max number of tries to deliver failed message before it
gives up

In the wild
Peerflix
PeerCDN
WebTorrent

In the wild
Peerflix
@mafintosh
!
PeerCDN
WebTorrent
@feross

IsWebTRwToCR MeaoddyeYest.com

ORTC
- WebRTC 1.1 (ORTC)
- SDP exchange not required
- Advanced stuff like simulcast
- More low level than WebRTC
- Shim for WebRTC to work on top of ORTC
- Important goal is compatibility. Existing
apps should not be affected

Communication

Opera Software
Cheers!
@shwetank
Fronteers
2014
shwetankd@opera.com