JooinK - DevFest Piemonte 2013

2,337 views

Published on

WebRTC, and most of HTML5 tecnologies as File API, WebGL, WebWorkers, ...,answer to a common problem, i.e. to make 'in browser' apps able to do operations impossible before, as access to the video and audio stream by a javascript api.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,337
On SlideShare
0
From Embeds
0
Number of Embeds
1,261
Actions
Shares
0
Downloads
24
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

JooinK - DevFest Piemonte 2013

  1. 1. WebRTC ... GWT & Browser Performance Alberto Mancini, Francesca Tosi JooinK.com
  2. 2. WebRTC Plug-in free realtime communication … WebRTC is a free, open project that enables web browsers with Real-Time Communications (RTC) capabilities via simple Javascript APIs. source: webrtc.org
  3. 3. WebRTC Just another JS API or WebRTC is a new front in the long war for an open and unencumbered web Brendan Eich (Mozilla CTO and inventor of JavaScript) ?
  4. 4. Browser Piercing Once upon a time ... LDP: Firewall Piercing mini-HOWTO François-René Rideau v0.97, 24 November 2001
  5. 5. Browser Piercing a Javascript Application cannot ➔ access the filesystem and then File API ➔ open full-duplex socket connection and then WebSockets
  6. 6. Browser Piercing a Javascript Application cannot ➔ use graphics accelerator and then WebGL ➔ span multiple threads and then WebWorkers
  7. 7. Browser Piercing a Javascript Application cannot ➔ acquire audio and video and then WebRTC ➔ communicate P2P and then WebRTC
  8. 8. WebRTC across platforms ● ● ● ● Chrome (mobile too) Firefox (mobile too) Opera Native Java and Obj-C
  9. 9. WebRTC JS-API Acquiring video and audio navigator.getUserMedia(constraints, successCallback, errorCallback);
  10. 10. WebRTC JS-API Acquiring video and audio navigator.getUserMedia(constraints, successCallback, errorCallback);
  11. 11. WebRTC JS-API Constraints ● Controls the contents of the MediaStream ● Media type, resolution, frame rate var constraints = {video: true}; video: { mandatory: { ... }, optional [{... }] }
  12. 12. WebRTC JS-API Constraints ● Controls the contents of the MediaStream ● Media type, resolution, frame rate var constraints = {video: true}; video: { mandatory: { chromeMediaSource: ‘screen’ }, optional [{... }] }
  13. 13. WebRTC JS-API Acquiring video and audio navigator.getUserMedia(constraints, successCallback, errorCallback);
  14. 14. WebRTC JS-API errorCallback function errorCallback(error) { console.log("error: ", error); } ;-)
  15. 15. WebRTC JS-API Acquiring video and audio navigator.getUserMedia(constraints, successCallback, errorCallback);
  16. 16. WebRTC JS-API successCallback function successCallback(stream) { ... } stream: MediaStream; a flux of audio- or video-related data.
  17. 17. WebRTC JS-API successCallback function successCallback(stream) { var video = ... video.src = window.URL.createObjectURL(stream); } (W3C File API)
  18. 18. WebRTC JS-API full sample var constraints = {video: true}; function successCallback(stream) { var video = document.querySelector("video"); video.src = window.URL.createObjectURL(stream); } function errorCallback(error) { console.log("navigator.getUserMedia error: ", error); } navigator.getUserMedia(constraints, successCallback, errorCallback);
  19. 19. WebRTC JS-API Do not forget ● to play the video ● to polyfill getUserMedia (moz*,webkit*,ms*)
  20. 20. WebRTC JS-API RTCPeerConnection → getUserMedia + RTCPeerConnection ←
  21. 21. WebRTC JS-API RTCDataChannel Bidirectional communication of arbitrary data ● low latency ● API similar to WebSockets
  22. 22. RTCPeerConnection sample pc = new RTCPeerConnection(null); pc.onaddstream = gotRemoteStream; pc.addStream(localStream); pc.createOffer(gotOffer); function gotOffer(desc) { pc.setLocalDescription(desc); sendOffer(desc); } function gotAnswer(desc) { pc.setRemoteDescription(desc); } function gotRemoteStream(e) { attachMediaStream(remoteVideo, e.stream); }
  23. 23. RTCPeerConnection
  24. 24. RTCPeerConnection .
  25. 25. RTCPeerConnection .
  26. 26. STUN .
  27. 27. TURN .
  28. 28. Yes BUT VideoChat already exists and we are too late to build skype without skype ... ● ● Video chat: ○ SimpleWebRTC ○ easyRTC ○ webRTC.io Peer-to-peer data: ○ PeerJS ○ Sharefest
  29. 29. Yes BUT ● getUserMedia: gives us real time stream from the cam ● we can draw the <video/> into a <canvas/> ● from the canvas we can extract pixel data getImageData() and then elaborate.
  30. 30. Computing (GWT 4 us) goals - Almost native performance - Manage complex algorithms and applications
  31. 31. Computing (GWT 4 us) goals - Almost native performance - Manage complex algorithms and applications our approach - javascript as a target language (GWT) - hand written optimized pieces of code
  32. 32. JooinK’s WebRTC sample Sample http://picshare.jooink.com Demo …
  33. 33. PicShare Tecnologies ● ● ● ● WebGL for 3D graphics WebRTC for MediaSteams ImgUR for pictures store AppEngine as a glue
  34. 34. FOSS gwt-nyartoolkit.googlecode.com gwt-webgl.googlecode.com The strategy: compile a java-lib with GWT into javascript
  35. 35. What’s next Challanges ● real-time on mobile apps ● real world recognition: computer vision (BoofCV)
  36. 36. What’s next Our approach ● use TypedArrays everywhere ● offload computation to the graphic accelerator through WebGL ● help the JIT compiler/optimizer by asm.js
  37. 37. That’s all folks!!! Alberto Mancini alberto@jooink.com
  38. 38. Markers

×