The State of WebRTC

12,447 views

Published on

WebRTC brings peer-to-peer networking to the browser, and it's here to stay. So what is WebRTC? How does it work? How do you use it? And what are others doing with it? In this talk, Rob covers the current state of WebRTC, outlines how to use it, and shows off some of the amazing things that it can do beyond video chat.

Published in: Technology
6 Comments
38 Likes
Statistics
Notes
No Downloads
Views
Total views
12,447
On SlideShare
0
From Embeds
0
Number of Embeds
492
Actions
Shares
0
Downloads
370
Comments
6
Likes
38
Embeds 0
No embeds

No notes for slide

The State of WebRTC

  1. 1. WEBRTCT H E S T A T E O F@ROBHAWKESHi, I’m Rob Hawkes and I’m here today to talk about the state of WebRTC.
  2. 2. ROB HAWKESWHO ISA little about me…- Digital Tinkerer (I play with new technologies and see how far I can push them)- Former Technical Evangelist at Mozilla (worked on HTML5 games, Firefox OS, and Web tech in general)- Author of multiple books, such as Foundation HTML5 Canvas- Currently taking time out to work on personal projects, such as…
  3. 3. VIZICITIES.COMViziCities, an experimental project that uses WebGL to bring real-world cities to life in the browser… in 3D!http://vizicities.com
  4. 4. @ROBHAWKESWHAT IS WEBRTC?What is WebRTC?Peer-to-peer communication technology for the browser.
  5. 5. @ROBHAWKESWEBRTC IS A FREE, OPENPROJECT THAT   ENABLES WEBBROWSERS WITH REAL-TIMEC O M M U N I C A T I O N S ( R T C )C A P A B I L I T I E S V I A S I M P L EJAVASCRIPT APIS.WEBRTC.ORG“WebRTC is a free, open project that enables web browsers with Real-Time Communications (RTC) capabilities viasimple Javascript APIs.”http://www.webrtc.org
  6. 6. @ROBHAWKESAT A BASIC LEVEL, WEBRTC ISLIKE SKYPE RUNNING IN THEBROWSER. SORT OF…ROB HAWKESAt a basic level, WebRTC is like Skype running in the browser.However, this is barely scratching the surface of what it can really do.
  7. 7. @ROBHAWKESWebRTC is a W3C specification, supported by Google, Mozilla, and Opera, amongst others.It’s currently in the editorial process and is not yet a full recommendation.The spec is a fantastic resource for in-depth technical information about WebRTC and how it should be implementedas per the browser. It’s a good read if you want to get right into the API structure.http://dev.w3.org/2011/webrtc/editor/webrtc.html
  8. 8. @ROBHAWKESThe official WebRTC site is also a fantastic resource for a slightly more human-readable overview of the technologyand the underlying architecture supporting it.The FAQ is also a good place to start when you have questions.http://www.webrtc.org/
  9. 9. @ROBHAWKESWEBRTCVSWEBSOCKETSNow we already have real-time communication technologies such as WebSockets, so why use WebRTC?At the the most basic level, WebSockets and WebRTC achieve different goals.WebSockets is purely about providing a reliable real-time data connection in JavaScript.On the other hand, WebRTC is about providing a network infrastructure for real-time media communication.
  10. 10. @ROBHAWKESWEBSOCKETSCommunication methods such as WebSockets require a server to act as a middleman, passing messages back andforth between users.
  11. 11. @ROBHAWKESWEBSOCKETSThe problem with a middleman approach is that you have to wait for your message to reach the server, be passedonto and reach the recipient, have a response sent back to the server, which in turn is then sent back to you.If both users had a latency to the server of 20ms, which is conservative, a round trip between the two users wouldtake 80ms. That’s nearly a tenth of a second, just ferrying messages back and forth.
  12. 12. @ROBHAWKESWEBRTCOn the other hand, WebRTC is a peer-to-peer connection directly between 2 users. No middleman.
  13. 13. @ROBHAWKESWEBRTCThis means you can send messages directly to the other user without hopping via a server.If the latency between the users was 20ms then a round-trip would take just 40ms, half the time it would take withsomething like WebSockets.
  14. 14. @ROBHAWKESUDPVSTCPAnother big draw to WebRTC is that it primarily uses UDP, a method for sending data across a network in anunreliable fashion.In comparison, technologies such as HTTP use TCP which is a reliable method of sending data.Without going into too much detail, UDP is a lot faster than TCP because it doesn’t worry about order or errorcorrection. Because of this you wouldn’t want to use it to send important data that needs to be received in order andwhole.Instead, UDP is commonly used in situations where you only care about the very latest piece of data and have noneed to wait for data sent in the past to be received correctly. VoIP and multiplayer games are perfect examples ofthis.WebRTC is the first time UDP has been available in the browser without plugins and because of this a lot ofdevelopers are excited to get their hands on it. I know I am!http://www.skullbox.net/tcpudp.phphttp://gafferongames.com/networking-for-game-programmers/udp-vs-tcp/
  15. 15. @ROBHAWKESWEBRTC USE CASESThe ability to connect peer-to-peer with another user certainly sounds cool, but what exactly is it useful for?
  16. 16. @ROBHAWKESVIDEO AND AUDIO CALLSThe most common use-case is video and audio calls, a la. Skype.This seems to be the approach most people are taking with WebRTC, which is fair considering that’s what it has beenmarketed as.But WebRTC is more than that…
  17. 17. @ROBHAWKESSCREEN SHARINGScreensharing is another approach, similar to the idea of sharing video.
  18. 18. @ROBHAWKESCOLLABORATIVE ACTIVITIESThere are also collaborative activities like writing or coding, this is another area which is gaining a lot of interest.
  19. 19. @ROBHAWKESMULTIPLAYER GAMESMultiplayer games are an area that will benefit hugely from WebRTC.This is mainly because of UDP and the ability to have unreliable network data connections, an approach used by thevast majority of multiplayer games outside of the Web.We also shouldn’t forget that the peer-to-peer nature of WebRTC will also allow for a new breed of Web games thatdon’t need servers to be played multiplayer. That will be very interesting to see progress.
  20. 20. @ROBHAWKESP2P FILE SHARINGA slightly more interesting use of the technology is peer-to-peer file sharing.The ability to quickly send a file directly to a friend without leaving the browser is certainly an appealing one.Also, imagine the possibility for something like BitTorrent in the browser!
  21. 21. @ROBHAWKESP2P CDNSomething I’d never considered before is the idea of using WebRTC to create a peer-to-peer CDN amongst yourwebsite visitors.The idea of share the serving of assets amongst a network of peer-to-peer connections is very intriguing.
  22. 22. @ROBHAWKESREMOTE CONTROLSomething that flips the common approach of WebRTC is the idea of remote control of devices from other devices.In this case, instead of connecting to another browser controlled by another person you are connecting two devicesowned by the same person.This way you can use one device to control the other without having to worry about a server or Internet connection.Theoretically this could work over the local network.One approach where this would be useful is for using mobile phones to act as a gamepad for games playing on adesktop or TV.
  23. 23. @ROBHAWKESWEBRTC APISThere are 3 main APIs involved in WebRTC.Let’s quickly take a look at each of them.
  24. 24. @ROBHAWKESMEDIASTREAMMediaStream API, commonly referred to as getUserMedia.This API gives you access to media data streams from devices such as webcams and microphones.http://dev.w3.org/2011/webrtc/editor/getusermedia.html
  25. 25. @ROBHAWKESGETUSERMEDIAIS NOT“WEBRTC SUPPORT”getUserMedia is not WebRTC.
  26. 26. @ROBHAWKESASCII Webcam
  27. 27. @ROBHAWKESPEERCONNECTIONPeerConnection API.This API lets you make peer-to-peer connections and attach media streams like video and audio.http://dev.w3.org/2011/webrtc/editor/webrtc.html#peer-to-peer-connections
  28. 28. @ROBHAWKESDATACHANNELDataChannel API.This API lets you send and receive arbitrary data across a peer-to-peer connection.http://dev.w3.org/2011/webrtc/editor/webrtc.html#peer-to-peer-data-api
  29. 29. @ROBHAWKESDESKTOPSUPPORTWe’ve seen what WebRTC offers, but what is the real-world support like?It’s still a very new technology but things are looking good, at least on desktop.
  30. 30. @ROBHAWKESFirefox supports WebRTC by default in version 22, which is currently going through the testing stages in the Nightlyand Aurora versions.If all goes well, functional WebRTC support will land in the public release of Firefox by the end of June.https://hacks.mozilla.org/2013/04/webrtc-update-our-first-implementation-will-be-in-release-soon-welcome-to-the-party-but-please-watch-your-head/
  31. 31. @ROBHAWKESThe current public release of Chrome has support for basic audio and video WebRTC calls enabled by default.DataChannel is supported in Chrome beta and is currently behind a flag.http://www.html5rocks.com/en/tutorials/webrtc/basics/http://updates.html5rocks.com/2013/02/WebRTC-data-channels-API-changes-and-Chrome-talks-to-Firefox
  32. 32. @ROBHAWKESOpera only supports the ability to get video and audio via getUserMedia.This is not really WebRTC support.
  33. 33. @ROBHAWKESIE has no WebRTC support natively but support can be added if you use Chrome Frame.Gotta love Chrome Frame!http://www.google.com/chromeframe
  34. 34. @ROBHAWKESMOBILESUPPORTWhile desktop support is pretty good, considering, mobile support isn’t so good.
  35. 35. @ROBHAWKESFirefox on Android has WebRTC support behind a config flag.It’s very early days and your mileage may vary while using it as the mobile implementation is changing quite rapidly.Mozilla plan to firm up their mobile WebRTC support in the coming months.http://www.morbo.org/2013/04/webrtc-support-on-android.html
  36. 36. @ROBHAWKESChrome on Android also has WebRTC support behind a config flag.It seems to be quite reliable and you can certainly make calls between Chrome mobile and a desktop browser. Verycool!https://groups.google.com/d/msg/discuss-webrtc/uFOMhd-AG0A/81p3dE_5peYJ
  37. 37. @ROBHAWKESOpera Mobile only has support for getUserMedia.Again, this is not really WebRTC support.
  38. 38. @ROBHAWKESINTEROPERABILITYBrowser support for the APIs is all well and good, but what is the interoperability like?It’s not very useful to have WebRTC support if you can’t make a call between different browsers and platforms.Up until recently the situation wasn’t very promising.http://www.webrtc.org/interop
  39. 39. However, earlier this year it was announced that a WebRTC connection can be made between public builds of Firefoxand Chrome.This was pretty big news and was co-announced on both the Chrome dev blog and the Mozilla Hacks blog. It was oneof the rare occurrences where the browser wars were put aside and true cooperation was embraced.http://blog.chromium.org/2013/02/hello-firefox-this-is-chrome-calling.htmlhttps://hacks.mozilla.org/2013/02/hello-chrome-its-firefox-calling/https://apprtc.appspot.com/
  40. 40. @ROBHAWKESWEBRTC DEMOIf all goes well, I’m going to show you a quick demo of WebRTC.http://freshtilledsoil.com/the-future-of-web/webrtc-video/https://apprtc.appspot.com
  41. 41. @ROBHAWKESUSING WEBRTCLet’s take a look at how to actually use WebRTC.
  42. 42. @ROBHAWKESMEDIASTREAMThe most basic API to look at first is the MediaStream API, specifically the getUserMedia method.
  43. 43. @ROBHAWKESvar constraints = {video: true};function successCallback(localMediaStream) {var video = document.querySelector("video");video.src = window.URL.createObjectURL(localMediaStream);video.play();}function errorCallback(error){console.log("getUserMedia error: ", error);}getUserMedia(constraints, successCallback, errorCallback);HTML5ROCKS.COMGETUSERMEDIA DEMOIn this example we have an implementation for accessing the webcam.It’s pretty basic but it gets the job done. The only other thing you need is a HTML video element to attach thewebcam stream to.It’s worth pointing out that, although you can’t see it, we’re making use of a simple polyfill to remove the (very few)browser inconsistencies in the API calls.You also need to run this through a domain, not the local filesystem, otherwise security restrictions will prevent youfrom accessing the media devices.http://www.simpl.info/getusermedia/https://code.google.com/p/webrtc-samples/source/browse/trunk/apprtc/js/adapter.js
  44. 44. @ROBHAWKESPEERCONNECTIONThings get a little more complicated when we look at the PeerConnection API, but once you understand the processit’s not so bad.We’ll first look at an example that makes a local PeerConnection call within a single browser page.It’s a good way to look at the code before adding in the extra functionality to make remote calls.
  45. 45. @ROBHAWKESfunction start() {getUserMedia({audio:true, video:true}, gotStream, function() {});}function gotStream(stream){vid1.src = window.URL.createObjectURL(stream);localstream = stream;}...HTML5ROCKS.COMPEERCONNECTION DEMO (1/4)In this example we’re again making use of the polyfill to avoid the browser prefix dance. We’re also missing theHTML as it’s most important to understand the JavaScript.1. The first step is to call getUserMedia2. Next, we connect the local stream to a HTML video element to you can see yourself3. Next, we store a reference to the local stream so we can access it later onhttp://www.simpl.info/rtcpeerconnection/https://code.google.com/p/webrtc-samples/source/browse/trunk/apprtc/js/adapter.js
  46. 46. @ROBHAWKES...function call() {var servers = null;window.pc1 = new RTCPeerConnection(servers);pc1.onicecandidate = iceCallback1;window.pc2 = new RTCPeerConnection(servers);pc2.onicecandidate = iceCallback2;pc2.onaddstream = gotRemoteStream;pc1.addStream(localstream);pc1.createOffer(gotDescription1);}...HTML5ROCKS.COMPEERCONNECTION DEMO (2/4)At this point we haven’t achieved anything more then the previous getUserMedia example.Once the webcam and audio is set up we can begin the process of making a PeerConnection.1. Firstly, we call the RTCPeerConnection method and pass it an empty variable - This variable can be used to define TURN and STUN servers, which can help when routing through firewalls2. The next step is to set up an event handler for onicecandidate - This is used to work out which methods work best for making a connection3. Next, we call RTCPeerConnection again and use this as our ‘remote’ connection4. We set up the onicecandidate handler for the remote connection5. And also set up the onaddstream handler for the remote connection - This lets you know when a remote stream has been received by PeerConnection6. Now we add the local media stream to the local PeerConnection7. Finally, we call the createOffer method of PeerConnection and pass a handler to deal with the returned sessiondescription. - We’ll look at the handler in more detail in a moment
  47. 47. @ROBHAWKESv=0o=- 5948426442422644553 2 IN IP4 127.0.0.1s=-t=0 0a=group:BUNDLE audio videoa=msid-semantic: WMS qcBqgcQpLthJWqiwDV4Hbujhpj0rTn5vvUZRm=audio 1 RTP/SAVPF 111 103 104 0 8 107 106 105 13 126...a=ssrc:2770545510 label:qcBqgcQpLthJWqiwDV4Hbujhpj0rTn5vvUZRv0SESSION DESCRIPTION EXAMPLEThe session description we get returned by createOffer is a method of describing the streaming media connectionthat we want to make between the two peers.This is just an example session description; what you see will vary depending on what you’re trying to do and yournetwork situation.http://www.html5rocks.com/en/tutorials/webrtc/basics/#toc-signalinghttp://en.wikipedia.org/wiki/Session_Description_Protocol
  48. 48. @ROBHAWKES...function iceCallback1(event){if (event.candidate) {pc2.addIceCandidate(new RTCIceCandidate(event.candidate));}}function iceCallback2(event){if (event.candidate) {pc1.addIceCandidate(new RTCIceCandidate(event.candidate));}}...HTML5ROCKS.COMPEERCONNECTION DEMO (3/4)Moving on; this is what the onicecandidate handlers look like, for both the local (top) and remote (bottom)connections.What’s happening here is that the returned connection methods (candidates) are added to the opposite connection sothey both know the best way to connect to the other.However, we’re still not ready to make the connection.
  49. 49. @ROBHAWKES...function gotDescription1(desc){pc1.setLocalDescription(desc);pc2.setRemoteDescription(desc);pc2.createAnswer(gotDescription2);}function gotDescription2(desc){pc2.setLocalDescription(desc);pc1.setRemoteDescription(desc);}function gotRemoteStream(e){vid2.src = window.URL.createObjectURL(e.stream);}HTML5ROCKS.COMPEERCONNECTION DEMO (4/4)The last chunk of code allows us to successfully hook up the PeerConnection and tidy up the loose ends.1. The first thing we do is set up the gotDescription1 handler, which is called by the createOffer method from earlier - This returns the session description for the local connection2. We then call the setLocalDescription method on the local PeerConnection3. We then call the setRemoteDescriotion method on the ‘remote’ PeerConnection - At this point we’re nearly ready to make the connection, we just need to respond to the offer4. The next step is to call createAnswer on the ‘remote’ PeerConnection and pass a handler for the returned sessiondescription.5. From here we jump into the gotDescription2 handler which deals with the session description for the ‘remote’PeerConnection created by createAnswer - Now it’s just a case of tying up the loose ends6. We call setLocalDescription on the ‘remote’ PeerConnection and pass it the session description7. We then call setRemoteDescription on the local PeerConnection and pass it the same session description - At this point you’ve successfully connected the two peers, but there is one last step…8. The last thing to do is to set up the gotRemoteStream handler - This is called when a media stream is added by a remote peer. Eg, when the connection starts. - All this does is set the stream, in this case a video, to another HTML video elementAnd you’re done!
  50. 50. @ROBHAWKESSIGNALLINGThe problem with the previous demo is that it only runs within a single browser page.Most of you who want to use WebRTC will want to make connections between remote browsers.A full code example of a remote WebRTC demo is beyond the scope of this talk, though I’ve linked to a greatexample in the slides.Instead, what we can do is briefly take a look at the concept of signalling… the discovery and connection of remotepeers.http://www.webrtc.org/demohttp://www.html5rocks.com/en/tutorials/webrtc/basics/#toc-signaling
  51. 51. @ROBHAWKESLOCAL PEERCONNECTIONWith the local PeerConnection example the act of peer discovery was not needed as the two peers were running fromthe same page and as such knew about each other already.
  52. 52. @ROBHAWKESDISCOVER PEERS VIA SERVER#ROOM1 #ROOM1With a full remote PeerConnection, the first step is to use a server to discover other peers (users) wanting to make aWebRTC connection. Ie. We use the server to work out which computers want to connect with each other as we can’tdo that locally.Common techniques include the use of a room name or a unique identifier that two peers can both use so the serverknows who to pair.Once peers are paired a handshake is performed so that each peer knows just enough information to connect to theother one. This is done by sharing ICE information and the session description.You’ve already done this with the local demo but it becomes a little more complicated when you do it remotely.Common approaches for this handshake use AJAX or WebSockets to send the ICE candidates and session descriptionsbetween peers via the server.To be clear the server is purely acting as a messaging proxy.And this is the only step that requires a server.http://blog.gingertech.net/2012/06/04/video-conferencing-in-html5-webrtc-via-web-sockets/http://www.html5rocks.com/en/tutorials/webrtc/basics/#toc-signaling-with-channelhttps://sites.google.com/site/muazkh/webrtc-order-the-code
  53. 53. @ROBHAWKESDIRECTLY CONNECT PEERSOnce the session description handshake has been made the two peers will be connected directly.At this point the server is no longer needed and you have a WebRTC connection up and running.It sounds quite simple in theory, and it kind of is thanks to the API introduced with WebRTC. The underlyingtechnology and networking to make this happen is much, much more complicated.
  54. 54. @ROBHAWKESDATACHANNELAt this point you may have a PeerConnection set up that is streaming webcam and microphone input between twobrowsers.If you want to then send arbitrary data over the PeerConnection you’ll need to set up a DataChannel.Let’s look at that next.
  55. 55. @ROBHAWKES...window.dc1 = pc1.createDataChannel("dataChannel1", {reliable: false});...pc2.ondatachannel = function(event) {window.dc2 = event.channel;dc2.onmessage = handleMessage;};var handleMessage = function(event) {console.log("DataChannel message received from pc1: n " + event.data);}...dc1.send("Hello, world!");DATACHANNEL DEMOWe’ll keep this one brief as it mostly relies on an existing PeerConnection, which you now know how to set up.In this example we make a call to the createDataChannel method on the local PeerConnection, passing it a label andsome option.We’re passing the reliable: false property in the options to use unreliable UDP for the DataChannel.We then set up a handler on the remote PeerConnection to listen for a DataChannel connection.When the DataChannel is received we store a reference to it and set up a handler to listen for incoming messages.From there, we can then call the send method on the local DataChannel to send messages to the remote peer.The DataChannel API is feels very similar to WebSockets.
  56. 56. @ROBHAWKESWEBRTC IN USENow that we’ve seen how WebRTC works, let’s take a look at some of the projects that are using it today.
  57. 57. @ROBHAWKESTowTruck by Mozilla.Collaborative browsing, editing, chat, and voice.Potential demo…https://towtruck.mozillalabs.com
  58. 58. @ROBHAWKESConversat.ioSmall video chat rooms.http://conversat.io/
  59. 59. @ROBHAWKESPeerJSAwesome cross-browser DataChannel framework.They also offer developers free signalling servers for your PeerJS apps, at least up to 50 concurrent connections.http://peerjs.com/
  60. 60. @ROBHAWKESTwelephoneTwitter-powered video and text chat.http://twelephone.com/
  61. 61. @ROBHAWKESTwilioIncreasingly-popular service that allows you to make and receive real-world phone calls programatically, includingfrom a browser with WebRTC.http://www.twilio.com/
  62. 62. @ROBHAWKESPeerCDNOffloading the serving of website assets to a peer-to-peer network of connected visitors.https://peercdn.com/
  63. 63. @ROBHAWKESPeerkitAnother approach at a peer-to-peer CDN.Potential demo…http://peerkit.com/
  64. 64. @ROBHAWKESBananaBreadPeer-to-peer multiplayer in a 3D WebGL game.https://hacks.mozilla.org/2013/03/webrtc-data-channels-for-great-multiplayer/
  65. 65. @ROBHAWKESExperimental sharing of browser tabs in Chrome.http://updates.html5rocks.com/2012/12/Screensharing-with-WebRTC
  66. 66. @ROBHAWKESPhonoA jQuery plugin for making phone calls from the browser.http://phono.com/webrtc
  67. 67. @ROBHAWKESAtt.jsEffectively powered by Phono, with a few extra bits and pieces.https://att.io/
  68. 68. @ROBHAWKESOpenTokAnother platform for creating audio and video calls within the browser using WebRTC.http://www.tokbox.com/
  69. 69. @ROBHAWKESFile HangoutExperiment with DataChannels for sharing files amongst multiple peers.https://webrtc-experiment.appspot.com/file-hangout/
  70. 70. @ROBHAWKESSqwiggleCollaborative dashboard for remote working, allowing you to see each other and share resources throughout the day.https://www.sqwiggle.com/
  71. 71. @ROBHAWKESEricsson Labs 3D WebRTC demoUsing output from a depth sensor synchronised with a peer-to-peer video call.https://labs.ericsson.com/blog/3d-webrtc-3d-video-communication-in-a-browser
  72. 72. @ROBHAWKESCodassiumCollaborative code editor, marketed towards programming interviews.http://codassium.com/
  73. 73. @ROBHAWKESGrimewire ‘OS’A crazy project aimed at creating a browser OS that uses WebRTC. I’m not even sure how yet, but it certainly soundscool!http://blog.grimwire.com/#2013-04-04-grimwire.md
  74. 74. @ROBHAWKESG O F O R T HAND WEBRTC!I hope that I’ve managed to pique your interest in WebRTC enough to start experimenting with it.There is so much that the technology can do that people just aren’t exploring yet, particularly around to use ofmobile devices.I’d love to see what you make with it so make sure to ping me on Twitter if you do have a play.
  75. 75. Tweetmap.itReal-time visualisation of tweetsViziCities.com3D visualisation of real-world citiesSlidesslideshare.net/robhawkesRawkes.comPersonal website and blogRECENT PROJECTS MORE COOL STUFF@robhawkesRob HawkesGet in touch with me on Twitter: @robhawkesFollow my blog (Rawkes) to keep up to date with stuff that I’m working on: http://rawkes.comViziCities http://vizicities.comTweetmap http://tweetmap.itThese slides are online at http://slideshare.net/robhawkes

×