The video for this presentation is located at: https://vimeo.com/133079307 Patrick Cason and Kenny House gave a talk to introduce Javascript developers to the basic concepts of WebRTC. In the talk are examples of how to implement WebRTC as well as a high-level overview of basic networking when streaming live audio and video peer-to-peer. To learn more about how Kenny and Patrick use WebRTC in their work, visit www.octovis.com.

1. WEBRTC IN JAVASCRIPT
REAL-TIME COMMUNICATION IN JAVASCRIPT
Introduction to WebRTC
1

2. WEBRTC IN JAVASCRIPT
IETF (RTCWEB SPECIFICATION)
WebRTC consists of a protocol specification and
a Javascript API specification that, if all are
implemented, will allow communication using
audio, video, and data sent along the most
direct possible path between participants.
2

3. WEBRTC IN JAVASCRIPT
History and Overview
• Been around since 2011, still in draft status, but it’s used
in production
• Google acquired two companies (On2… video codecs
and GIPS… VoIP calling)
• Wrote a Javascript API for submission to W3C and
called it WebRTC (“Web Real-time Communication”)
3

4. WEBRTC IN JAVASCRIPT
Why we chose WebRTC…
• The web is moving towards real-time functionality
• People want something native to their browser
• Video and audio calls on the web !== Flash or ActiveX
4
(AND WHY YOU SHOULD TOO)

5. WEBRTC IN JAVASCRIPT
1. Have media to send (webcam, microphone, etc.)
2. Figure out best way for peers to communicate
3. Determine your media’s format (SDP)
4. Send communication and media information
5. Repeat steps 1 - 4 for other peers
5

6. WEBRTC IN JAVASCRIPT
1. Have media to send (webcam, microphone, etc.)
2. Figure out best way for peers to communicate
3. Determine your media’s format (SDP)
4. Send communication and media information
5. Repeat steps 1 - 4 for other peers
6

7. WEBRTC IN JAVASCRIPT
getUserMedia()
• Handles all transport of
video and audio
information from the
hardware to the web
• Each MediaStream
contains multiple
“tracks”
7
navigator.getUserMedia()

8. WEBRTC IN JAVASCRIPT
getUserMedia()
var constraints = { video: true };
function successCallback(stream) {
var video = document.querySelector('video');
video.src = window.URL.createObjectURL(stream);
}
function errorCallback(error) {
console.log('navigator.getUserMedia error: ', error);
}
navigator.getUserMedia(constraints, successCallback, errorCallback);
8

9. WEBRTC IN JAVASCRIPT
getUserMedia() constraints
• Intended to control the media stream via a
configuration object
• Controls media type, resolution, frame rate, etc.
• Able to specify specs as either “mandatory” or
“optional”
• Constraints only affect local view of your stream
9

10. WEBRTC IN JAVASCRIPT
getUserMedia() audio analysis
• Only works for Chrome local audio streams, you can’t
currently analyze remote audio streams
10
// Success callback when requesting audio input stream
function gotStream(stream) {
var audioContext = new webkitAudioContext();
// Create an AudioNode from the stream
var mediaStreamSource = audioContext.createMediaStreamSource(stream);
// Connect it to the destination or any other node for processing!
mediaStreamSource.connect(audioContext.destination);
}
navigator.webkitGetUserMedia({ audio: true }, gotStream);

11. WEBRTC IN JAVASCRIPT
1. Have media to send (webcam, microphone, etc.)
2. Figure out best way for peers to communicate
3. Determine your media’s format (SDP)
4. Send communication and media information
5. Repeat steps 1 - 4 for other peers
11

12. WEBRTC IN JAVASCRIPT
TCP & UDP
• UDP (User Datagram Protocol)
Unreliable… fast, and with a low overhead
• TCP (Transmission Control Protocol)
Reliable… slow, and with a high overhead
• HTTP uses TCP for connections, vast majority of web
services use TCP
12

13. WEBRTC IN JAVASCRIPT
All about dat UDP
• UDP is a better choice for streaming media…
• It’s fast
• You don’t usually need every single packet
• Caveat? You need to know where to send packets.
13

14. WEBRTC IN JAVASCRIPT
Expectations
14

15. WEBRTC IN JAVASCRIPT
Reality
15

16. WEBRTC IN JAVASCRIPT
… ICE BABY? TOO COLD?
But then there’s ICE
16

17. WEBRTC IN JAVASCRIPT
ICE, ICE Connections…
• ICE is a priority-based protocol for testing connections
between two computers
• ICE calls these possible connection points “candidates”
• Computers send them to each other, test them out, and
choose the best fit
17

18. WEBRTC IN JAVASCRIPT
STUN
18

19. WEBRTC IN JAVASCRIPT
TURN
19

20. WEBRTC IN JAVASCRIPT
ICE Connections
var pc;
pc = new RTCPeerConnection({
'iceServers': [
{ 'url': 'stun:stun.services.mozilla.com' },
{ 'url': 'stun:stun.l.google.com:19302' }
]
});
pc.onicecandidate = function(event) {
if(event.candidate !== null) {
mySocket.send(JSON.stringify({
'ice': event.candidate
}));
}
}
20

21. WEBRTC IN JAVASCRIPT
Gotchas
• In addition to your signaling system server, you’ll also
need to run your own STUN/TURN server
• ICE checks for local network connections first, so it will
almost always work over a local network
• This doesn’t mean it works over the Internet
• Particularly embarrassing when demoing for a
client…
21

22. WEBRTC IN JAVASCRIPT 22
WAT.

23. WEBRTC IN JAVASCRIPT
1. Have media to send (webcam, microphone, etc.)
2. Figure out best way for peers to communicate
3. Determine your media’s format (SDP)
4. Send communication and media information
5. Repeat steps 1 - 4 for other peers
23

24. WEBRTC IN JAVASCRIPT
SDP
• Two peers on ICE communicate via a spec called SDP
(“Session Description Protocol”) which tell what each
peer is interested in
• Do you want audio? Sure.
• Do you want video? Definitely.
• Breakfast in bed? Every Sunday.
24

25. WEBRTC IN JAVASCRIPT
SDP (continued)
• SDP also handles codec specifications
• Generally no need to modify an SDP string
• First peer sends an offer, second peer sends an answer
• Both ICE and SDP require a signaling server to set up
the direct connection between peers
25

26. WEBRTC IN JAVASCRIPT
Let’s analyze an SDP string…
type: offer, sdp: v=0
o=- 581856173060444966 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio video
a=msid-semantic: WMS 5BgnHND1rh5IsIJXBGN0EnzD0xoGWD8Hd901
m=audio 9 RTP/SAVPF 111 103 104 9 0 8 126
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:JI6Xwt9wq/v5HuY7
a=ice-pwd:eNXo/5Sx1JtHCAciusSzmZzO
a=fingerprint:sha-256 A0:DE:
39:65:10:33:02:44:C8:14:57:62:28:89:D9:45:69:DA:B6:58:44:A8:D8:2A:BC:
46:D3:E0:2B:D1:0D:E4
a=setup:actpass
a=mid:audio
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=sendrecv
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=fmtp:111 minptime=10; useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:126 telephone-event/8000
a=maxptime:60
a=ssrc:2786781455 cname:Raffm+GJz1SzeA5+
a=ssrc:2786781455 msid:5BgnHND1rh5IsIJXBGN0EnzD0xoGWD8Hd901
2c9da2ce-219e-4e89-9ad5-1eaada3d79c8
a=ssrc:2786781455 mslabel:5BgnHND1rh5IsIJXBGN0EnzD0xoGWD8Hd901
a=ssrc:2786781455 label:2c9da2ce-219e-4e89-9ad5-1eaada3d79c8
m=video 9 RTP/SAVPF 100 116 117 96
26
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:JI6Xwt9wq/v5HuY7
a=ice-pwd:eNXo/5Sx1JtHCAciusSzmZzO
a=fingerprint:sha-256 A0:DE:
39:65:10:33:02:44:C8:14:57:62:28:89:D9:45:69:DA:B6:58:44:A8:D8:2A:BC:
46:D3:E0:2B:D1:0D:E4
a=setup:actpass
a=mid:video
a=extmap:2 urn:ietf:params:rtp-hdrext:toffset
a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:4 urn:3gpp:video-orientation
a=sendrecv
a=rtcp-mux
a=rtpmap:100 VP8/90000
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=rtcp-fb:100 goog-remb
a=rtpmap:116 red/90000
a=rtpmap:117 ulpfec/90000
a=rtpmap:96 rtx/90000
a=fmtp:96 apt=100
a=ssrc-group:FID 1802804178 3581727714
a=ssrc:1802804178 cname:Raffm+GJz1SzeA5+
a=ssrc:1802804178 msid:5BgnHND1rh5IsIJXBGN0EnzD0xoGWD8Hd901 168833a1-b0f1-439a-
be29-5687f1d8072e
a=ssrc:1802804178 mslabel:5BgnHND1rh5IsIJXBGN0EnzD0xoGWD8Hd901
a=ssrc:1802804178 label:168833a1-b0f1-439a-be29-5687f1d8072e
a=ssrc:3581727714 cname:Raffm+GJz1SzeA5+
a=ssrc:3581727714 msid:5BgnHND1rh5IsIJXBGN0EnzD0xoGWD8Hd901 168833a1-b0f1-439a-
be29-5687f1d8072e
a=ssrc:3581727714 mslabel:5BgnHND1rh5IsIJXBGN0EnzD0xoGWD8Hd901
a=ssrc:3581727714 label:168833a1-b0f1-439a-be29-5687f1d8072e

27. WEBRTC IN JAVASCRIPT 27
WAT.

28. WEBRTC IN JAVASCRIPT
RTCPeerConnection
pc = new RTCPeerConnection(null);
pc.addStream(localStream);
pc.createOffer(sendingOffer);
function sendingOffer(desc) {
pc.setLocalDescription(desc);
sendAnswer(desc);
}
// When answer received over signaling
function gotAnswer(desc) {
pc.setRemoteDescription(desc);
}
28

29. WEBRTC IN JAVASCRIPT
Security on WebRTC
29

30. WEBRTC IN JAVASCRIPT
1. Have media to send (webcam, microphone, etc.)
2. Figure out best way for peers to communicate
3. Determine your media’s format (SDP)
4. Send communication and media information
5. Repeat steps 1 - 4 for other peers
30

31. WEBRTC IN JAVASCRIPT
HTTP://PASTEBIN.COM/RK9IZITX
Walkthrough
31

32. WEBRTC IN JAVASCRIPT
Platform Support
• Google Chrome (mostly)
• Chrome for Android (mostly)
• Firefox (kinda)
• Opera (not really)
• Safari & IE (srsly bro?)
• Native Java and Objective-C bindings (about the same as
Chrome)
32

33. WEBRTC IN JAVASCRIPT
Nerdy Details
• Video codec support: VP8, H264
• Audio codec support: G.711, iLBC, iSAC, OPUS
• Media transport protocol: RTP/RTCP
• Security protocol: SRTP
• NAT traversal: STUN/TURN/ICE
• Signaling: Peer-to-peer, but requires a signaling component
(agnostic) to notify peers of status changes and new messages
33

34. WEBRTC IN JAVASCRIPT
adapter.js & Temasys
• adapter.js (maintained by Google) is a Javascript shim
made to normalize browser differences
• Minimizes the effects of spec churn from subtleties
• Temasys also makes their own version of adapter.js
which adds a Flash fallback for a poor man’s browsers
34

35. WEBRTC IN JAVASCRIPT
WebRTC Internals
• God’s gift to WebRTC
developers
• Available at:
chrome://webrtc-internals
• Or programmatically:
myPeerConnection.getStats()
35

36. WEBRTC IN JAVASCRIPT
Resources
• Getting Started with WebRTC
• Capturing Audio & Video in HTML5
• WebRTC in the Real World
• Google I/O Presentation Video
• HTML5 WebRTC Explained Video
• WebRTC @ MDN
• WebRTC Book
36

37. WEBRTC IN JAVASCRIPT
TELEMEDICINE PLATFORM USING WEBRTC
About Octovis
37

38. WEBRTC IN JAVASCRIPT
IOS? API’S? WEARABLES?
We’re Hiring!
38

39. WEBRTC IN JAVASCRIPT
OCTOTALK.IO
Check out a working demo!
39

40. WEBRTC IN JAVASCRIPT
CONTACT@OCTOVIS.COM
Interested in getting Octotalk technology
in your company?
40

41. WEBRTC IN JAVASCRIPT
Thanks.
41
PATRICK CASON
Front-end Engineer & Designer
KENNY HOUSE
Software Engineer