Advertisement
Advertisement

More Related Content

Advertisement
Advertisement

WebRTC: A front-end perspective

  1. WebRTC A front-end perspective Opera Software Fronteers 2014 @shwetank
  2. About me Shwetank Dixit Opera Software Fronteers 2014 @shwetank Developer Relations, Opera Software
  3. http://www.thehindu.com/news/national/other-states/using-pcos-in-mumbai-show-id-proof/article4241422.ece
  4. Source: https://www.flickr.com/photos/rambow/858719954
  5. Source: https://www.flickr.com/photos/rambow/858717314 Source: https://www.flickr.com/photos/rambow/858719954
  6. What is WebRTC? Real Time Peer-to-Peer Communication
  7. but … So far I’ve seen very few web people talking about it
  8. Whats the biggest barrier to WebRTC adoption?* Lack of Awareness Not supported by MS Not supported by Apple Other 0% 5.5% 11% 16.5% 22% * WebRTC World Outlook Survey 2014
  9. WebRTC is for but … Web Developers
  10. Mediastream
  11. Mediastream RTCPeerConnection
  12. Mediastream RTCPeerConnection RTCDataChannel
  13. mediaStream + ` ` Device Camera Microphone
  14. getUserMedia + ` ` Device Camera Microphone
  15. Sqwiggle
  16. gUM + CSS(Filters, Masks, Blend Modes) Create interesting visual effects from camera output in real-time using CSS
  17. gUM + Web Audio Sound powered actions and navigation
  18. World’s first webcam http://en.wikipedia.org/wiki/Trojan_Room_coffee_pot
  19. What can you achieve with gUM? - Motion Detection (Baby Monitors, Security Cam) - Hotspots - Gesture Recognition - Emotion Recognition - Sound Detection and Visualisation - Voice / Sound Activated Controls - Capture video (Kind of) (whammy.js, gifshot)
  20. gUM: Current way to do it navigator.getUserMedia* *with prefixes
  21. gUM: Broken Error Handling Exceptions are not caught by the current API. Promises can fix this. Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Sep/0171.html
  22. gUM: Current Way var failure = function(reason){ log("Failed to show camera: " + reason.message); } ! navigator.getUserMedia(constraints, function(stream) { try { $video.src = window.URL.createObjectURL(stream); $video.play(); } catch(e) { failure(e); } //failure must tolerate non-MediaStreamError }, failure); Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Sep/0171.html
  23. gUM: In the future ? navigator.mediaDevices.getUserMedia
  24. gUM: In the future ? Promises Proposal navigator.mediaDevices.getUserMedia(constraints) .then( function(stream) { videoElement.src = window.URL.createObjectURL(stream); videoElement.play(); }).catch(function (reason){ log(“Failed to show camera: ” +reason.message); }); Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Sep/0171.html
  25. Debate This is still up for debate. Some are eager to just ship v1.0 and add support for promises in gUM at a later stage. ! Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Oct/0003.html
  26. gUM: Other concerns Impossible for a gUM app to use the device flash for low-light ! Use (auto) zoom Use (auto) focus
  27. gUM: Other concerns “If you look further into this API then it doesn’t offer any advanced features that you expect in a good camera app: focus, flash, zoom ` etc. It is clear that …(getUserMedia).. has been designed for use in P2P applications and not as a dedicated camera experience.” - Paul Kinlan
  28. Mediastream Image-Capture ! Will provide a deeper level of control for taking photos ! w3.org/TR/image-capture/
  29. Mediastream Image-Capture White Balance ISO Red-Eye Reduction Exposure Brightness Contrast ` Saturation Sharpness Zoom takePhoto()
  30. Mediastream Image-Capture Focus? Flash? ¯_(ツ)_/¯
  31. Getting permission is not easy Some users don’t seem to notice the permissions dialog in certain browsers
  32. Getting permission is not easy Some users don’t seem to notice the permissions dialog in certain browsers
  33. Some users don’t seem to notice the permissions dialog in certain browsers
  34. Some users don’t seem to notice the permissions dialog in certain browsers
  35. Some users don’t seem to notice the permissions dialog in certain browsers
  36. Getting permission is not easy Permissions dialogs are different on different platforms and browsers … ` especially mobile
  37. HTTPS helps … If the app is on HTTPS, then the browser will remember the last granted permission.
  38. RTCPeerConnection
  39. webrtcglossary.com
  40. RTCPeerConnection
  41. ICE - Interactive Connectivity Establishment * May or may not wax a chump like a candle.
  42. STUN Whats my external IP Address? Your external IP Address is x.x.x.x
  43. TURN around… Total eclipse of the (p2p) call …
  44. TURN Basically relays stuff from one party to the other.
  45. Big Fat TURN Server
  46. TURN Basically relays stuff from one party to the other. In this case, connection won’t be p2p, but at least the connection will be made!
  47. TURN Basically relays stuff from one party to the other. In this case, connection won’t be p2p, but at least the connection will be made! Generally speaking, a TURN server will include a STUN server too.
  48. Free ICE Project github.com/DamonOehlman/freeice (List of free STUN/TURN servers to use in your projects)
  49. The WebRTC Signalling Process WebSocket || XHR || SIP || Carrier Pigeon
  50. Session Description (SDP) v=0 o=- 680121471469462884 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE audio video a=msid-semantic: WMS GUKF430Khp9jEQiPrdYe0LbTAALiNAKAIfl2 Audio Lines m=audio 54278 RTP/SAVPF 111 103 104 0 8 106 105 13 126 c=IN IP4 180.6.6.6 a=rtcp:54278 IN IP4 180.6.6.6 ! ICE Candidates a=candidate:4022866446 1 udp 2113937151 192.168.0.197 36768 typ host generation 0 a=candidate:4022866446 2 udp 2113937151 192.168.0.197 36768 typ host generation 0 a=candidate:2706108158 1 tcp 1509957375 192.168.0.197 0 typ host generation 0 a=candidate:2706108158 2 tcp 1509957375 192.168.0.197 0 typ host generation 0 a=candidate:1853887674 1 udp 1845501695 46.2.2.2 36768 typ srflx raddr 192.168.0.197 rport 36768 generation 0 a=candidate:1853887674 2 udp 1845501695 46.2.2.2 36768 typ srflx raddr 192.168.0.197 rport 36768 generation 0 a=candidate:2157334355 1 udp 33562367 180.6.6.6 54278 typ relay raddr 46.2.2.2 rport 38135 generation 0 a=candidate:2157334355 2 udp 33562367 180.6.6.6 54278 typ relay raddr 46.2.2.2 rport 38135 generation 0
  51. The WebRTC Signalling Process WebSocket || XHR || SIP || Carrier Pigeon Creates Offer local description
  52. The WebRTC Signalling Process WebSocket || XHR || SIP || Carrier Pigeon Receives Offer local description
  53. The WebRTC Signalling Process WebSocket || XHR || SIP || Carrier Pigeon Receives Offer remote description local description
  54. The WebRTC Signalling Process WebSocket || XHR || SIP || Carrier Pigeon Creates and sends Answer local description local description remote description
  55. The WebRTC Signalling Process WebSocket || XHR || SIP || Carrier Pigeon Receives Answer local description local description remote description
  56. The WebRTC Signalling Process WebSocket || XHR || SIP || Carrier Pigeon Receives Answer local description local description remote description remote description
  57. The WebRTC Signalling Process WebSocket || XHR || SIP || Carrier Pigeon Connectivity Checks local description local description remote description remote description
  58. The WebRTC Signalling Process WebSocket || XHR || SIP || Carrier Pigeon P2P Connection Established
  59. Libraries
  60. Bandwidth Numbers ! 720p : 1-2 Mbps // VP8 at 30fps 360p : 0.5 - 1 Mbps // VP8 at 30fps 180p : 0.1 - 0.5 Mbps // VP8 at 30fps Audio : .06 - 0.51 Mbps // Opus
  61. Debugging opera://webrtc-internals chrome://webrtc-internals about:webrtc ! The getStats() method ! callstats.io !
  62. Data Channels Once RTCPeerConnection is established, we can use Data Channels to transfer data P2P. Data can be text or binary data
  63. Whats the advantage? Data Channels provide a * High Performance * P2P * Low Latency Connection to others
  64. Working with Data datachannel .onopen // Data Channel opened .onclose // Data Channel closed .onerror // Error occurred .onmessage // New message arrived .send() // Send a message
  65. DataChannels can work exactly like WebSockets if you want, but they don’t have to …
  66. SCTP // Transport Protocol DTLS // Encryption UDP
  67. notable options ordered send data packets in ordered fashion or not ! maxRetransmitTime Max time to try to send a failed message ! maxRetransmits Max number of tries to deliver failed message before it gives up
  68. In the wild Peerflix PeerCDN WebTorrent
  69. In the wild Peerflix @mafintosh ! PeerCDN WebTorrent @feross
  70. IsWebTRwToCR MeaoddyeYest.com
  71. ORTC - WebRTC 1.1 (ORTC) - SDP exchange not required - Advanced stuff like simulcast - More low level than WebRTC - Shim for WebRTC to work on top of ORTC - Important goal is compatibility. Existing apps should not be affected
  72. Communication
  73. Opera Software Cheers! @shwetank Fronteers 2014 shwetankd@opera.com
Advertisement