0
WebRTC
... GWT & Browser Performance
Alberto Mancini, Francesca Tosi
JooinK.com
WebRTC
Plug-in free realtime communication …

WebRTC is a free, open project that enables web browsers
with Real-Time Comm...
WebRTC
Just another JS API
or

WebRTC is a new front in the long war for an
open and unencumbered web
Brendan Eich (Mozill...
Browser Piercing
Once upon a time ...

LDP: Firewall Piercing mini-HOWTO
François-René Rideau
v0.97, 24 November 2001
Browser Piercing
a Javascript Application cannot

➔ access the filesystem
and then File API

➔ open full-duplex socket con...
Browser Piercing
a Javascript Application cannot

➔ use graphics accelerator
and then WebGL

➔ span multiple threads
and t...
Browser Piercing
a Javascript Application cannot

➔ acquire audio and video
and then WebRTC

➔ communicate P2P
and then We...
WebRTC across platforms
●
●
●
●

Chrome (mobile too)
Firefox (mobile too)
Opera
Native Java and Obj-C
WebRTC JS-API
Acquiring video and audio

navigator.getUserMedia(constraints,
successCallback,
errorCallback);
WebRTC JS-API
Acquiring video and audio
navigator.getUserMedia(constraints,
successCallback,
errorCallback);
WebRTC JS-API
Constraints
● Controls the contents of the MediaStream
● Media type, resolution, frame rate
var constraints ...
WebRTC JS-API
Constraints
● Controls the contents of the MediaStream
● Media type, resolution, frame rate
var constraints ...
WebRTC JS-API
Acquiring video and audio

navigator.getUserMedia(constraints,
successCallback,
errorCallback);
WebRTC JS-API
errorCallback

function errorCallback(error) {
console.log("error: ", error);
}

;-)
WebRTC JS-API
Acquiring video and audio

navigator.getUserMedia(constraints,
successCallback,
errorCallback);
WebRTC JS-API
successCallback
function successCallback(stream) {
...
}
stream: MediaStream; a flux of audio- or video-rela...
WebRTC JS-API
successCallback
function successCallback(stream) {
var video = ...
video.src =
window.URL.createObjectURL(st...
WebRTC JS-API
full sample
var constraints = {video: true};
function successCallback(stream) {
var video = document.querySe...
WebRTC JS-API
Do not forget
● to play the video
● to polyfill getUserMedia (moz*,webkit*,ms*)
WebRTC JS-API
RTCPeerConnection
→
getUserMedia
+
RTCPeerConnection
←
WebRTC JS-API
RTCDataChannel

Bidirectional communication of arbitrary data
● low latency
● API similar to WebSockets
RTCPeerConnection sample
pc = new RTCPeerConnection(null);
pc.onaddstream = gotRemoteStream;
pc.addStream(localStream);
pc...
RTCPeerConnection
RTCPeerConnection
.
RTCPeerConnection
.
STUN
.
TURN
.
Yes BUT
VideoChat already exists and we are too late to build skype without
skype ...
●

●

Video chat:
○ SimpleWebRTC
○ e...
Yes BUT
● getUserMedia: gives us real time stream from the cam
● we can draw the <video/> into a <canvas/>
● from the canv...
Computing (GWT 4 us)
goals
- Almost native performance
- Manage complex algorithms and applications
Computing (GWT 4 us)
goals
- Almost native performance
- Manage complex algorithms and applications
our approach
- javascr...
JooinK’s WebRTC sample
Sample
http://picshare.jooink.com
Demo …
PicShare
Tecnologies
●
●
●
●

WebGL for 3D graphics
WebRTC for MediaSteams
ImgUR for pictures store
AppEngine as a glue
FOSS
gwt-nyartoolkit.googlecode.com
gwt-webgl.googlecode.com
The strategy: compile a java-lib with GWT into javascript
What’s next
Challanges

● real-time on mobile apps
● real world recognition: computer vision
(BoofCV)
What’s next
Our approach

● use TypedArrays everywhere
● offload computation to the graphic
accelerator through WebGL
● he...
That’s all folks!!!

Alberto Mancini
alberto@jooink.com
Markers
Upcoming SlideShare
Loading in...5
×

JooinK - DevFest Piemonte 2013

2,018

Published on

WebRTC, and most of HTML5 tecnologies as File API, WebGL, WebWorkers, ...,answer to a common problem, i.e. to make 'in browser' apps able to do operations impossible before, as access to the video and audio stream by a javascript api.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,018
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
23
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "JooinK - DevFest Piemonte 2013"

  1. 1. WebRTC ... GWT & Browser Performance Alberto Mancini, Francesca Tosi JooinK.com
  2. 2. WebRTC Plug-in free realtime communication … WebRTC is a free, open project that enables web browsers with Real-Time Communications (RTC) capabilities via simple Javascript APIs. source: webrtc.org
  3. 3. WebRTC Just another JS API or WebRTC is a new front in the long war for an open and unencumbered web Brendan Eich (Mozilla CTO and inventor of JavaScript) ?
  4. 4. Browser Piercing Once upon a time ... LDP: Firewall Piercing mini-HOWTO François-René Rideau v0.97, 24 November 2001
  5. 5. Browser Piercing a Javascript Application cannot ➔ access the filesystem and then File API ➔ open full-duplex socket connection and then WebSockets
  6. 6. Browser Piercing a Javascript Application cannot ➔ use graphics accelerator and then WebGL ➔ span multiple threads and then WebWorkers
  7. 7. Browser Piercing a Javascript Application cannot ➔ acquire audio and video and then WebRTC ➔ communicate P2P and then WebRTC
  8. 8. WebRTC across platforms ● ● ● ● Chrome (mobile too) Firefox (mobile too) Opera Native Java and Obj-C
  9. 9. WebRTC JS-API Acquiring video and audio navigator.getUserMedia(constraints, successCallback, errorCallback);
  10. 10. WebRTC JS-API Acquiring video and audio navigator.getUserMedia(constraints, successCallback, errorCallback);
  11. 11. WebRTC JS-API Constraints ● Controls the contents of the MediaStream ● Media type, resolution, frame rate var constraints = {video: true}; video: { mandatory: { ... }, optional [{... }] }
  12. 12. WebRTC JS-API Constraints ● Controls the contents of the MediaStream ● Media type, resolution, frame rate var constraints = {video: true}; video: { mandatory: { chromeMediaSource: ‘screen’ }, optional [{... }] }
  13. 13. WebRTC JS-API Acquiring video and audio navigator.getUserMedia(constraints, successCallback, errorCallback);
  14. 14. WebRTC JS-API errorCallback function errorCallback(error) { console.log("error: ", error); } ;-)
  15. 15. WebRTC JS-API Acquiring video and audio navigator.getUserMedia(constraints, successCallback, errorCallback);
  16. 16. WebRTC JS-API successCallback function successCallback(stream) { ... } stream: MediaStream; a flux of audio- or video-related data.
  17. 17. WebRTC JS-API successCallback function successCallback(stream) { var video = ... video.src = window.URL.createObjectURL(stream); } (W3C File API)
  18. 18. WebRTC JS-API full sample var constraints = {video: true}; function successCallback(stream) { var video = document.querySelector("video"); video.src = window.URL.createObjectURL(stream); } function errorCallback(error) { console.log("navigator.getUserMedia error: ", error); } navigator.getUserMedia(constraints, successCallback, errorCallback);
  19. 19. WebRTC JS-API Do not forget ● to play the video ● to polyfill getUserMedia (moz*,webkit*,ms*)
  20. 20. WebRTC JS-API RTCPeerConnection → getUserMedia + RTCPeerConnection ←
  21. 21. WebRTC JS-API RTCDataChannel Bidirectional communication of arbitrary data ● low latency ● API similar to WebSockets
  22. 22. RTCPeerConnection sample pc = new RTCPeerConnection(null); pc.onaddstream = gotRemoteStream; pc.addStream(localStream); pc.createOffer(gotOffer); function gotOffer(desc) { pc.setLocalDescription(desc); sendOffer(desc); } function gotAnswer(desc) { pc.setRemoteDescription(desc); } function gotRemoteStream(e) { attachMediaStream(remoteVideo, e.stream); }
  23. 23. RTCPeerConnection
  24. 24. RTCPeerConnection .
  25. 25. RTCPeerConnection .
  26. 26. STUN .
  27. 27. TURN .
  28. 28. Yes BUT VideoChat already exists and we are too late to build skype without skype ... ● ● Video chat: ○ SimpleWebRTC ○ easyRTC ○ webRTC.io Peer-to-peer data: ○ PeerJS ○ Sharefest
  29. 29. Yes BUT ● getUserMedia: gives us real time stream from the cam ● we can draw the <video/> into a <canvas/> ● from the canvas we can extract pixel data getImageData() and then elaborate.
  30. 30. Computing (GWT 4 us) goals - Almost native performance - Manage complex algorithms and applications
  31. 31. Computing (GWT 4 us) goals - Almost native performance - Manage complex algorithms and applications our approach - javascript as a target language (GWT) - hand written optimized pieces of code
  32. 32. JooinK’s WebRTC sample Sample http://picshare.jooink.com Demo …
  33. 33. PicShare Tecnologies ● ● ● ● WebGL for 3D graphics WebRTC for MediaSteams ImgUR for pictures store AppEngine as a glue
  34. 34. FOSS gwt-nyartoolkit.googlecode.com gwt-webgl.googlecode.com The strategy: compile a java-lib with GWT into javascript
  35. 35. What’s next Challanges ● real-time on mobile apps ● real world recognition: computer vision (BoofCV)
  36. 36. What’s next Our approach ● use TypedArrays everywhere ● offload computation to the graphic accelerator through WebGL ● help the JIT compiler/optimizer by asm.js
  37. 37. That’s all folks!!! Alberto Mancini alberto@jooink.com
  38. 38. Markers
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×