Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

WebRTC + Socket.io: building a skype-like video chat with native javascript

19,782 views

Published on

Presentation of my last talk at MilanoJS event. It is the case history of a project where we used the WebRTC to make a chat skype style, using also WebSockets with Socket.io on NodeJS

Published in: Software

WebRTC + Socket.io: building a skype-like video chat with native javascript

  1. 1. WEBRTC + SOCKET.IO BUILDING A SKYPE-LIKE VIDEO CHAT WITH NATIVE JAVASCRIPT /MICHELE DI SALVATORE @MIKDISAL Javascript Architect @ Objectway
  2. 2. Audio and video communication and peer-to-peer data sharing through a web application Native javascript (no plugins) Open source appear.in
  3. 3. HOW IS BORN Global IP Solutions acquired by Google in 2010 Google open sourced the technologies
  4. 4. SUPPORTED BROWSERS AND PLATFORMS
  5. 5. IE AND SAFARI? freeTemasys Plugin
  6. 6. JAVASCRIPT WEBRTC API MediaStream RTCPeerConnection RTCDataChannel
  7. 7. MEDIASTREAM (AKA GETUSERMEDIA) Acquiring audio and video Can contain multiple 'tracks' var constraints = { video: { mandatory: { minWidth: 640, minHeight: 360 } }; function successCallback(stream) { var video = document.querySelector("video"); video.src = window.URL.createObjectURL(stream); } function errorCallback(error) { console.log("navigator.getUserMedia error: ", error); } Constraints controls the contents of the MediaStream
  8. 8. RTCPEERCONNECTION Communicating audio and video Signal processing Codec handling Peer to peer communication Security Bandwidth management ...
  9. 9. RTCDATACHANNEL Communicating arbitrary data Same API as WebSockets Ultra-low latency Unreliable or reliable Secure
  10. 10. SIGNALING Exchanging 'session description' objects What formats I support, what I want to send Network information for peer-to-peer setup Any message mechanism and protocol
  11. 11. JSEP JavaScript Session Establishment Protocol
  12. 12. RTCSESSIONDESCRIPTION EXAMPLE v=0 o=- 7614219274584779017 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE audio video a=msid-semantic: WMS m=audio 1 RTP/SAVPF 111 103 104 0 8 107 106 105 13 126 c=IN IP4 0.0.0.0 a=rtcp:1 IN IP4 0.0.0.0 a=ice-ufrag:W2TGCZw2NZHuwlnf a=ice-pwd:xdQEccP40E+P0L5qTyzDgfmW a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level a=mid:audio a=rtcp-mux a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:9c1AHz27dZ9xPI91YNfSlI67/EMkjHHIHORiC a=rtpmap:111 opus/48000/2 ...
  13. 13. P2P IN REAL WORLD NAT & Firewalls?
  14. 14. STUN SERVER Gives my public IP address Simple server, cheap to run Data flows peer-to-peer
  15. 15. STUN SERVER
  16. 16. TURN SERVER Fallback for p2p failing Data via server Call works everywhere
  17. 17. STUN + TURN SERVERS
  18. 18. ICE Interactive Connectivity Establishment A framework to connect peers via UDP, enabling RTCPeerConnection via STUN and TURN server
  19. 19. WEBRTC SECURITY On Chrome only via https Encryption for media and data Sandboxed
  20. 20. WEBRTC LIBRARIES & CO. by by by by ... adapter.js webrtc rtc-everywhere contra SimpleWebRTC &Yet RTCMultiConnection Muaz Khan
  21. 21. RTCMULTICONNECTION A javascript wrapper library supporting approximately all possible peer-to-peer features. WEBRTC EXPERIMENTS Tons of open source experiments based on RTCMultiConnection: webrtc-experiment.com
  22. 22. WEBSOCKETS Standardized in 2011 Interactive communication session between browser and server Event driven Sending strings and binary data
  23. 23. WEBSOCKETS BROWSER SUPPORT All and IE10+ ajax polling as fallback
  24. 24. SOCKET.IO A popular open source library with client and server implementations Server side written in Node and easy to start var server = require('http').createServer(); var io = require('socket.io')(server); io.on('connection', function(socket){ socket.on('hi!', function(data){ socket.emit('a-message', {hello: 'world'}); }); socket.on('disconnect', function(){}); }); server.listen(3000);
  25. 25. SOCKET.IO CLIENT var socket = io.connect('http://localhost:3000/'); socket.on('connect', function () { socket.emit('hi!'); }); socket.on('a-message', function (data) { console.log(data.hello); });
  26. 26. GOAL A SKYPE LIKE CHAT
  27. 27. MAIN ISSUES WebRTC architecture like IRC rooms with shared chat: text, video and data direct chat: only one to one room managed by initiator conversation type can change without agreement: text to video
  28. 28. IMPLEMENTATION
  29. 29. STEP 1 UserA is logged in and is connected to the socket server
  30. 30. STEP 2 The app sends a "presence" event to other users through the socket The server forwards the message only to user's contacts Each present user replies with the same "presence" event
  31. 31. SOCKET.IO FORWARD ROLE
  32. 32. STEP 3 Each present user replies with the same "presence" event
  33. 33. STEP 4 When a user receives a "presence" event, it automatically knows that the sender is online Also the server sends the “presence” event to all, when a user socket is disconnected
  34. 34. STEP 5 UserA starts a conversation and the app creates the webrtc connection
  35. 35. STEP 6 UserA sends a message via webrtc
  36. 36. STEP 7 UserB receives a notification
  37. 37. STEP 8 UserB starts writing a message
  38. 38. STEP 9 UserA receives the "is typing" info sent via webrtc
  39. 39. STEP 10 UserA starts a video call sending a request to UserB via socket
  40. 40. STEP 11 UserB receives the request and accepting starts a video call via webrtc
  41. 41. STEP 12 The 2 users can video chat via webrtc Each user can close the video at any time or start a text chat
  42. 42. STEP 13 A user can send text messages also
  43. 43. STEP 14 A user can start an audio chat in the same way as a video chat
  44. 44. chat BEHIND THE SCENES
  45. 45. WHERE IS WEBRTC? Behind this actions chat messages video / audio call "is typing" info
  46. 46. AND THE OTHERS? via WebSockets presence video / audio initialization all status infos
  47. 47. THAT'S ALL! THANK YOU ALL /Michele Di Salvatore @MikDiSal

×