1.
WebRTC
A front-end perspective
Opera Software
Fronteers
2014
@shwetank

2.
About me
Shwetank Dixit
Opera Software
Fronteers
2014
@shwetank
Developer Relations, Opera Software

3.
http://www.thehindu.com/news/national/other-states/using-pcos-in-mumbai-show-id-proof/article4241422.ece

4.
Source: https://www.flickr.com/photos/rambow/858719954

5.
Source: https://www.flickr.com/photos/rambow/858717314 Source: https://www.flickr.com/photos/rambow/858719954

6.
What is
WebRTC?
Real Time
Peer-to-Peer
Communication

7.
but …
So far I’ve seen very
few web people
talking about it

8.
Whats the biggest barrier to WebRTC adoption?*
Lack of Awareness
Not supported by MS
Not supported by Apple
Other
0% 5.5% 11% 16.5% 22%
* WebRTC World Outlook Survey 2014

9.
WebRTC is for
but … Web Developers

10.
Mediastream

11.
Mediastream
RTCPeerConnection

12.
Mediastream
RTCPeerConnection
RTCDataChannel

13.
mediaStream
+
` `
Device Camera Microphone

14.
getUserMedia
+
` `
Device Camera Microphone

15.
Sqwiggle

16.
gUM + CSS(Filters, Masks, Blend Modes)
Create interesting visual effects from
camera output in real-time using CSS

17.
gUM + Web Audio
Sound powered actions
and navigation

18.
World’s first webcam
http://en.wikipedia.org/wiki/Trojan_Room_coffee_pot

19.
What can you achieve with gUM?
- Motion Detection (Baby Monitors, Security Cam)
- Hotspots
- Gesture Recognition
- Emotion Recognition
- Sound Detection and Visualisation
- Voice / Sound Activated Controls
- Capture video (Kind of) (whammy.js, gifshot)

20.
gUM: Current way to do it
navigator.getUserMedia*
*with prefixes

21.
gUM: Broken Error Handling
Exceptions are not caught by the current API.
Promises can fix this.
Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Sep/0171.html

22.
gUM: Current Way
var failure = function(reason){
log("Failed to show camera: " + reason.message);
}
!
navigator.getUserMedia(constraints, function(stream) {
try {
$video.src = window.URL.createObjectURL(stream);
$video.play();
} catch(e) { failure(e); } //failure must tolerate non-MediaStreamError
}, failure);
Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Sep/0171.html

23.
gUM: In the future ?
navigator.mediaDevices.getUserMedia

24.
gUM: In the future ? Promises Proposal
navigator.mediaDevices.getUserMedia(constraints)
.then( function(stream) {
videoElement.src = window.URL.createObjectURL(stream);
videoElement.play();
}).catch(function (reason){
log(“Failed to show camera: ” +reason.message);
});
Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Sep/0171.html

25.
Debate
This is still up for debate.
Some are eager to just ship v1.0 and add support for
promises in gUM at a later stage.
!
Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Oct/0003.html

26.
gUM: Other concerns
Impossible for a gUM app to
use the device flash for low-light
!
Use (auto) zoom
Use (auto) focus

27.
gUM: Other concerns
“If you look further into this API then it
doesn’t offer any advanced features
that you expect in a good camera app:
focus, flash, zoom `
etc. It is clear that
…(getUserMedia).. has been designed
for use in P2P applications and not as a
dedicated camera experience.”
- Paul Kinlan

28.
Mediastream Image-Capture
!
Will provide a deeper level of control
for taking photos
!
w3.org/TR/image-capture/

29.
Mediastream Image-Capture
White Balance
ISO
Red-Eye Reduction
Exposure
Brightness
Contrast
`
Saturation
Sharpness
Zoom
takePhoto()

30.
Mediastream Image-Capture
Focus? Flash? ¯_(ツ)_/¯

31.
Getting permission is not easy
Some users don’t seem to
notice the permissions dialog
in certain browsers

32.
Getting permission is not easy
Some users don’t seem to
notice the permissions dialog
in certain browsers

33.
Some users don’t seem to
notice the permissions dialog
in certain browsers

34.
Some users don’t seem to
notice the permissions dialog
in certain browsers

35.
Some users don’t seem to
notice the permissions dialog
in certain browsers

36.
Getting permission is not easy
Permissions dialogs are different on
different platforms and browsers …
`
especially mobile

37.
HTTPS helps …
If the app is on HTTPS, then the
browser will remember the last
granted permission.

38.
RTCPeerConnection

39.
webrtcglossary.com

40.
RTCPeerConnection

41.
ICE - Interactive Connectivity Establishment
* May or may not wax a chump like a candle.

42.
STUN
Whats my
external IP Address?
Your external
IP Address is x.x.x.x

43.
TURN around…
Total eclipse of the (p2p) call …

44.
TURN
Basically relays stuff from one
party to the other.

45.
Big Fat TURN Server

46.
TURN
Basically relays stuff from one
party to the other.
In this case, connection won’t
be p2p, but at least the
connection will be made!

47.
TURN
Basically relays stuff from one
party to the other.
In this case, connection won’t
be p2p, but at least the
connection will be made!
Generally speaking, a TURN server will
include a STUN server too.

48.
Free ICE Project
github.com/DamonOehlman/freeice
(List of free STUN/TURN servers to use in your projects)

49.
The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon

50.
Session Description (SDP)
v=0
o=- 680121471469462884 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio video
a=msid-semantic: WMS GUKF430Khp9jEQiPrdYe0LbTAALiNAKAIfl2
Audio Lines
m=audio 54278 RTP/SAVPF 111 103 104 0 8 106 105 13 126
c=IN IP4 180.6.6.6
a=rtcp:54278 IN IP4 180.6.6.6
!
ICE Candidates
a=candidate:4022866446 1 udp 2113937151 192.168.0.197 36768 typ host generation 0
a=candidate:4022866446 2 udp 2113937151 192.168.0.197 36768 typ host generation 0
a=candidate:2706108158 1 tcp 1509957375 192.168.0.197 0 typ host generation 0
a=candidate:2706108158 2 tcp 1509957375 192.168.0.197 0 typ host generation 0
a=candidate:1853887674 1 udp 1845501695 46.2.2.2 36768 typ srflx raddr 192.168.0.197 rport 36768 generation 0
a=candidate:1853887674 2 udp 1845501695 46.2.2.2 36768 typ srflx raddr 192.168.0.197 rport 36768 generation 0
a=candidate:2157334355 1 udp 33562367 180.6.6.6 54278 typ relay raddr 46.2.2.2 rport 38135 generation 0
a=candidate:2157334355 2 udp 33562367 180.6.6.6 54278 typ relay raddr 46.2.2.2 rport 38135 generation 0

51.
The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Creates Offer
local description

52.
The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Receives Offer
local description

53.
The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Receives Offer
remote description
local description

54.
The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Creates and
sends Answer
local description
local description
remote description

55.
The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Receives Answer
local description
local description
remote description

56.
The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Receives Answer
local description
local description
remote description
remote description

57.
The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Connectivity
Checks
local description
local description
remote description
remote description

58.
The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
P2P Connection Established

59.
Libraries

60.
Bandwidth Numbers
!
720p : 1-2 Mbps // VP8 at 30fps
360p : 0.5 - 1 Mbps // VP8 at 30fps
180p : 0.1 - 0.5 Mbps // VP8 at 30fps
Audio : .06 - 0.51 Mbps // Opus

61.
Debugging
opera://webrtc-internals
chrome://webrtc-internals
about:webrtc
!
The getStats() method
!
callstats.io
!

62.
Data Channels
Once RTCPeerConnection is
established, we can use Data Channels
to transfer data P2P.
Data can be text or binary data

63.
Whats the advantage?
Data Channels provide a
* High Performance
* P2P
* Low Latency
Connection to others

64.
Working with Data
datachannel
.onopen // Data Channel opened
.onclose // Data Channel closed
.onerror // Error occurred
.onmessage // New message arrived
.send() // Send a message

65.
DataChannels can work
exactly like WebSockets if
you want, but they don’t
have to …

66.
SCTP // Transport Protocol
DTLS // Encryption
UDP

67.
notable options
ordered
send data packets in ordered fashion or not
!
maxRetransmitTime
Max time to try to send a failed message
!
maxRetransmits
Max number of tries to deliver failed message before it
gives up

68.
In the wild
Peerflix
PeerCDN
WebTorrent

69.
In the wild
Peerflix
@mafintosh
!
PeerCDN
WebTorrent
@feross

70.
IsWebTRwToCR MeaoddyeYest.com

71.
ORTC
- WebRTC 1.1 (ORTC)
- SDP exchange not required
- Advanced stuff like simulcast
- More low level than WebRTC
- Shim for WebRTC to work on top of ORTC
- Important goal is compatibility. Existing
apps should not be affected

72.
Communication

73.
Opera Software
Cheers!
@shwetank
Fronteers
2014
shwetankd@opera.com