Downloaded 88 times
More Related Content
WebRTC: A front-end perspective
- 1. WebRTC A front-endperspective Opera Software Fronteers 2014 @shwetank
- 2. About me ShwetankDixit Opera Software Fronteers 2014 @shwetank Developer Relations, Opera Software
- 3.
- 4.
- 5. Source: https://www.flickr.com/photos/rambow/858717314 Source:https://www.flickr.com/photos/rambow/858719954
- 8. What is WebRTC? Real Time Peer-to-Peer Communication
- 10. but … Sofar I’ve seen very few web people talking about it
- 11. Whats the biggestbarrier to WebRTC adoption?* Lack of Awareness Not supported by MS Not supported by Apple Other 0% 5.5% 11% 16.5% 22% * WebRTC World Outlook Survey 2014
- 12. WebRTC is for but … Web Developers
- 13.
- 14.
- 15.
- 16. mediaStream + `` Device Camera Microphone
- 17. getUserMedia + `` Device Camera Microphone
- 18.
- 19. gUM + CSS(Filters,Masks, Blend Modes) Create interesting visual effects from camera output in real-time using CSS
- 20. gUM + WebAudio Sound powered actions and navigation
- 22. World’s first webcam http://en.wikipedia.org/wiki/Trojan_Room_coffee_pot
- 23. What can youachieve with gUM? - Motion Detection (Baby Monitors, Security Cam) - Hotspots - Gesture Recognition - Emotion Recognition - Sound Detection and Visualisation - Voice / Sound Activated Controls - Capture video (Kind of) (whammy.js, gifshot)
- 24. gUM: Current wayto do it navigator.getUserMedia* *with prefixes
- 25. gUM: Broken ErrorHandling Exceptions are not caught by the current API. Promises can fix this. Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Sep/0171.html
- 26. gUM: Current Way var failure = function(reason){ log("Failed to show camera: " + reason.message); } ! navigator.getUserMedia(constraints, function(stream) { try { $video.src = window.URL.createObjectURL(stream); $video.play(); } catch(e) { failure(e); } //failure must tolerate non-MediaStreamError }, failure); Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Sep/0171.html
- 27. gUM: In thefuture ? navigator.mediaDevices.getUserMedia
- 28. gUM: In thefuture ? Promises Proposal navigator.mediaDevices.getUserMedia(constraints) .then( function(stream) { videoElement.src = window.URL.createObjectURL(stream); videoElement.play(); }).catch(function (reason){ log(“Failed to show camera: ” +reason.message); }); Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Sep/0171.html
- 29. Debate This isstill up for debate. Some are eager to just ship v1.0 and add support for promises in gUM at a later stage. ! Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Oct/0003.html
- 30. gUM: Other concerns Impossible for a gUM app to use the device flash for low-light ! Use (auto) zoom Use (auto) focus
- 31. gUM: Other concerns “If you look further into this API then it doesn’t offer any advanced features that you expect in a good camera app: focus, flash, zoom ` etc. It is clear that …(getUserMedia).. has been designed for use in P2P applications and not as a dedicated camera experience.” - Paul Kinlan
- 32. Mediastream Image-Capture ! Will provide a deeper level of control for taking photos ! w3.org/TR/image-capture/
- 33. Mediastream Image-Capture WhiteBalance ISO Red-Eye Reduction Exposure Brightness Contrast ` Saturation Sharpness Zoom takePhoto()
- 34. Mediastream Image-Capture Focus?Flash? ¯_(ツ)_/¯
- 35. Getting permission isnot easy Some users don’t seem to notice the permissions dialog in certain browsers
- 36. Getting permission isnot easy Some users don’t seem to notice the permissions dialog in certain browsers
- 37. Some users don’tseem to notice the permissions dialog in certain browsers
- 38. Some users don’tseem to notice the permissions dialog in certain browsers
- 39. Some users don’tseem to notice the permissions dialog in certain browsers
- 40. Getting permission isnot easy Permissions dialogs are different on different platforms and browsers … ` especially mobile
- 42. HTTPS helps … If the app is on HTTPS, then the browser will remember the last granted permission.
- 43.
- 46.
- 47.
- 48. ICE - InteractiveConnectivity Establishment * May or may not wax a chump like a candle.
- 49. STUN Whats my external IP Address? Your external IP Address is x.x.x.x
- 50. TURN around… Totaleclipse of the (p2p) call …
- 51. TURN Basically relaysstuff from one party to the other.
- 52. Big Fat TURNServer
- 53. TURN Basically relaysstuff from one party to the other. In this case, connection won’t be p2p, but at least the connection will be made!
- 54. TURN Basically relaysstuff from one party to the other. In this case, connection won’t be p2p, but at least the connection will be made! Generally speaking, a TURN server will include a STUN server too.
- 55. Free ICE Project github.com/DamonOehlman/freeice (List of free STUN/TURN servers to use in your projects)
- 56. The WebRTC SignallingProcess WebSocket || XHR || SIP || Carrier Pigeon
- 57. Session Description (SDP) v=0 o=- 680121471469462884 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE audio video a=msid-semantic: WMS GUKF430Khp9jEQiPrdYe0LbTAALiNAKAIfl2 Audio Lines m=audio 54278 RTP/SAVPF 111 103 104 0 8 106 105 13 126 c=IN IP4 180.6.6.6 a=rtcp:54278 IN IP4 180.6.6.6 ! ICE Candidates a=candidate:4022866446 1 udp 2113937151 192.168.0.197 36768 typ host generation 0 a=candidate:4022866446 2 udp 2113937151 192.168.0.197 36768 typ host generation 0 a=candidate:2706108158 1 tcp 1509957375 192.168.0.197 0 typ host generation 0 a=candidate:2706108158 2 tcp 1509957375 192.168.0.197 0 typ host generation 0 a=candidate:1853887674 1 udp 1845501695 46.2.2.2 36768 typ srflx raddr 192.168.0.197 rport 36768 generation 0 a=candidate:1853887674 2 udp 1845501695 46.2.2.2 36768 typ srflx raddr 192.168.0.197 rport 36768 generation 0 a=candidate:2157334355 1 udp 33562367 180.6.6.6 54278 typ relay raddr 46.2.2.2 rport 38135 generation 0 a=candidate:2157334355 2 udp 33562367 180.6.6.6 54278 typ relay raddr 46.2.2.2 rport 38135 generation 0
- 58. The WebRTC SignallingProcess WebSocket || XHR || SIP || Carrier Pigeon Creates Offer local description
- 59. The WebRTC SignallingProcess WebSocket || XHR || SIP || Carrier Pigeon Receives Offer local description
- 60. The WebRTC SignallingProcess WebSocket || XHR || SIP || Carrier Pigeon Receives Offer remote description local description
- 61. The WebRTC SignallingProcess WebSocket || XHR || SIP || Carrier Pigeon Creates and sends Answer local description local description remote description
- 62. The WebRTC SignallingProcess WebSocket || XHR || SIP || Carrier Pigeon Receives Answer local description local description remote description
- 63. The WebRTC SignallingProcess WebSocket || XHR || SIP || Carrier Pigeon Receives Answer local description local description remote description remote description
- 64. The WebRTC SignallingProcess WebSocket || XHR || SIP || Carrier Pigeon Connectivity Checks local description local description remote description remote description
- 65. The WebRTC SignallingProcess WebSocket || XHR || SIP || Carrier Pigeon P2P Connection Established
- 66.
- 70. Bandwidth Numbers ! 720p : 1-2 Mbps // VP8 at 30fps 360p : 0.5 - 1 Mbps // VP8 at 30fps 180p : 0.1 - 0.5 Mbps // VP8 at 30fps Audio : .06 - 0.51 Mbps // Opus
- 71. Debugging opera://webrtc-internals chrome://webrtc-internals about:webrtc ! The getStats() method ! callstats.io !
- 72. Data Channels OnceRTCPeerConnection is established, we can use Data Channels to transfer data P2P. Data can be text or binary data
- 73. Whats the advantage? Data Channels provide a * High Performance * P2P * Low Latency Connection to others
- 74. Working with Data datachannel .onopen // Data Channel opened .onclose // Data Channel closed .onerror // Error occurred .onmessage // New message arrived .send() // Send a message
- 75. DataChannels can work exactly like WebSockets if you want, but they don’t have to …
- 76. SCTP // TransportProtocol DTLS // Encryption UDP
- 77. notable options ordered send data packets in ordered fashion or not ! maxRetransmitTime Max time to try to send a failed message ! maxRetransmits Max number of tries to deliver failed message before it gives up
- 78. In the wild Peerflix PeerCDN WebTorrent
- 79. In the wild Peerflix @mafintosh ! PeerCDN WebTorrent @feross
- 80.
- 84. ORTC - WebRTC1.1 (ORTC) - SDP exchange not required - Advanced stuff like simulcast - More low level than WebRTC - Shim for WebRTC to work on top of ORTC - Important goal is compatibility. Existing apps should not be affected
- 85.
- 86. Opera Software Cheers! @shwetank Fronteers 2014 shwetankd@opera.com