Unconventional webapps with  GWT/Elemental, WebRTC &          WebGL                     Alberto Mancini                   ...
GWT 2.5 ElementalElemental is a new library for fast, lightweight,and "to the metal" web programming in GWT.Elemental incl...
Unconventional... insomma qualcosa che non ci sembravapossibile fare con il solo browser, tipo accederea devices attaccati...
WebRTCWebRTC (Web Real-Time Communication) is an APIdefinition being drafted by W3C and IETF.The goal of WebRTC is to enab...
WebRTC (with Elemental)Video in a canvas     final VideoElement videoElement =                        Browser.getDocument(...
WebRTC (with Elemental)binding <video/> to userMedia (search for facelogin on googlecode)public void bindVideoToUserMedia(...
WebGLWebGL (Web Graphics Library) is a JavaScript API forrendering interactive 3D graphics and 2D graphics withinany compa...
WebGL (with Elemental)Video in a canvas (3d rendering context)  ...  WebGLRenderingContext ctx3d =        (WebGLRenderingC...
WebGL (with Elemental)  shadersVertex Shader                                          Fragment Shaderprecision mediump flo...
WebGL (with Elemental)  shadersFragment Shader                                        Fragment Shader...// Sepia tone     ...
WebGL (with Elemental)  3DWebGL e una libreria 2D !!Trasformazioni prospettiche (proiezioni), frustum, modelviewvanno impl...
WebGL (with Elemental)    3Ddouble a = (right + left) / (right - left);double b = (top + bottom) / (top - bottom);double c...
WebGL (with Elemental)  3D/Wavefront-OBJLoading Wavefront-obj files ...parser in java facile da trovare in reteClientBundl...
UnconventionalTutto quello che abbiamo visto fino ad ora si poteva fare,probabilmente con lo stesso sforzo, direttamente i...
GWTCompiler: Java -> JavascriptJavascript as a target language
NyARToolkitARToolKit is a computer tracking library for creation of strong augmentedreality applications that overlay virt...
NyARToolkit con GWThttp://jooink.blogpsot.comsuper-source tagper InputStream & altre parti della JRE non disponibili in GW...
How It Works   cam    WebRTC      <video/>                                 canvas    ImageData                            ...
Demo http://www.jooink.com/experiments/DevFest2012
Upcoming SlideShare
Loading in …5
×

Unconventional webapps with gwt:elemental & html5

2,387
-1

Published on

Talk di Alberto Mancini alla DevFest 2012 @ Firenze

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,387
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Unconventional webapps with gwt:elemental & html5

  1. 1. Unconventional webapps with GWT/Elemental, WebRTC & WebGL Alberto Mancini alberto@jooink.comBlog: http://jooink.blogspot.comOnline Demo: http://www.jooink.com/experiments/DevFest2012
  2. 2. GWT 2.5 ElementalElemental is a new library for fast, lightweight,and "to the metal" web programming in GWT.Elemental includes every HTML5 feature, ... DOM access... WebGL, WebAudio, WebSockets, WebRTC, WebIntents, Shadow DOM, the File API, and more.... we generate Java code directly from the WebIDL filesused by JavaScript engines. (like DART)
  3. 3. Unconventional... insomma qualcosa che non ci sembravapossibile fare con il solo browser, tipo accederea devices attaccati al client, una webcamper esempio.Lidea e farsi una foto ...
  4. 4. WebRTCWebRTC (Web Real-Time Communication) is an APIdefinition being drafted by W3C and IETF.The goal of WebRTC is to enable applications such asvoice calling, video chat and P2P file sharing withoutplugins.Specification & Up to date info: http://www.webrtc.orgSupport: Chrome 22+, ... well work-in-progress at least for firefox.what we need: getUserMedia (chrome !!)
  5. 5. WebRTC (with Elemental)Video in a canvas final VideoElement videoElement = Browser.getDocument().createVideoElement(); final CanvasElement canvas = Browser.getDocument().createCanvasElement(); final CanvasRenderingContext2D ctx2d = (CanvasRenderingContext2D)canvas.getContext("2d"); ... //repeatedly, e.g. in a Timer or RepeatingCommand ctx2d.drawImage(videoElement, 0, 0);Take snapshot as easy as: Image img = new Image(canvas.toDataURL("png"));
  6. 6. WebRTC (with Elemental)binding <video/> to userMedia (search for facelogin on googlecode)public void bindVideoToUserMedia(final VideoElement video, final EventListener l) { final Mappable map = (Mappable) JsMappable.createObject(); map.setAt("video", true); Browser.getWindow().getNavigator().webkitGetUserMedia(map, new NavigatorUserMediaSuccessCallback() { public boolean onNavigatorUserMediaSuccessCallback (LocalMediaStream stream) { setVideoSrc(video, stream); video.play(); ... }, new NavigatorUserMediaErrorCallback() {...}); }private static native void setVideoSrc( VideoElement v, LocalMediaStream s) /*-{ v.src = window.webkitURL.createObjectURL(s);}-*/;
  7. 7. WebGLWebGL (Web Graphics Library) is a JavaScript API forrendering interactive 3D graphics and 2D graphics withinany compatible web browser without the use of plug-ins.Specification: http://www.khronos.org/webgl/Support: http://caniuse.com/webglChrome 22+, FireFox 15+, Safari 6+, Opera 12+ (partial)
  8. 8. WebGL (with Elemental)Video in a canvas (3d rendering context) ... WebGLRenderingContext ctx3d = (WebGLRenderingContext)canvas.getContext("experimental-webgl");Drawing is a bit harder: create a texture, create a background rectangle, use the video as a texture ctx3d.texImage2D(WebGLRenderingContext.TEXTURE_2D, 0, WebGLRenderingContext.RGBA, WebGLRenderingContext.RGBA, WebGLRenderingContext.UNSIGNED_BYTE, videoElement);
  9. 9. WebGL (with Elemental) shadersVertex Shader Fragment Shaderprecision mediump float; precision mediump float;attribute vec2 a_position; uniform sampler2D u_image;attribute vec2 a_texCoord; varying vec2 v_texCoord;uniform vec2 u_resolution;varying vec2 v_texCoord; void main() {void main() { gl_FragColor = vec2 zeroToOne = a_position / u_resolution; texture2D(u_image, v_texCoord); vec2 zeroToTwo = zeroToOne * 2.0; } vec2 clipSpace = zeroToTwo - 1.0; gl_Position = vec4(clipSpace * vec2(1, -1), 1, 1); v_texCoord = a_texCoord;} from html5rocks.coms WebGL Fundamentals ... probably :o
  10. 10. WebGL (with Elemental) shadersFragment Shader Fragment Shader...// Sepia tone precision mediump float;// Convert to greyscale using NTSC weightings uniform sampler2D u_image; float grey = dot(texture2D(u_image, uniform sampler2D SECOND_u_image; v_texCoord).rgb, vec3(0.299, 0.587, 0.114)); uniform float alpha; gl_FragColor = varying vec2 v_texCoord; vec4(grey * vec3(1.2, 1.0, 0.8), 1.0); void main(void) {... vec4 prev = texture2D(SECOND_u_image,// Negative vec2(v_texCoord.s,1.0-v_texCoord.t)); vec4 texColour = vec4 cur = texture2D(u_image, v_texCoord); texture2D(u_image, v_texCoord); gl_FragColor = alpha*prev + (1.0-alpha)*cur; gl_FragColor = vec4(1.0 - texColour.rgb, 1.0); }from http://r3dux.org/2011/06/glsl-image-processing/
  11. 11. WebGL (with Elemental) 3DWebGL e una libreria 2D !!Trasformazioni prospettiche (proiezioni), frustum, modelviewvanno implementate.VertexShader: applichera le trasformazioni (matrici) che noi gliforniremo, vertice per verticeFragmentShader: usera, pixel-per-pixel, le informazionicalcolate vertice-per-vertice (ed interpolate dal sistema)per assegnare il colore
  12. 12. WebGL (with Elemental) 3Ddouble a = (right + left) / (right - left);double b = (top + bottom) / (top - bottom);double c = (farPlane + nearPlane) / (farPlane - nearPlane);double d = (2 * farPlane * nearPlane) / (farPlane - nearPlane);double x = (2 * nearPlane) / (right - left);double y = (2 * nearPlane) / (top - bottom);double[] perspectiveMatrix = new double[]{ x, 0, a, 0, 0, y, b, 0, 0, 0, c, d, 0, 0, -1, 0 };....ctx3d.uniformMatrix4fv(pMatrixUniform, false, Utilities.createArrayOfFloat32(perspectiveMatrix));...private native static Float32Array createFloat32Array(JsArrayOfNumber a) /*-{ return new Float32Array(a);}-*/;
  13. 13. WebGL (with Elemental) 3D/Wavefront-OBJLoading Wavefront-obj files ...parser in java facile da trovare in reteClientBundle DataResource perfetto per il loadingasincrono dei modelliJavascript per calcolare le normali se non sono nelmodello
  14. 14. UnconventionalTutto quello che abbiamo visto fino ad ora si poteva fare,probabilmente con lo stesso sforzo, direttamente in javascript. Ma usare GWT ci da o no una marcia in piu ?Nota:non stiamo dicendo che quello che segue non si potesse fare injavascript, ndr: Turing completeness, se ce ne fosse bisogno !
  15. 15. GWTCompiler: Java -> JavascriptJavascript as a target language
  16. 16. NyARToolkitARToolKit is a computer tracking library for creation of strong augmentedreality applications that overlay virtual imagery on the real world.NyARToolKit is an ARToolkit class library released for virtual machines,particularly those that host Java, C# and Android.Up to date infohttp://nyatla.jp/nyartoolkit/wp/?page_id=198Support:Java ... "Write once, run anywhere" (WORA), or sometimes write once, runeverywhere (WORE) ... in my browser ?!?!?!
  17. 17. NyARToolkit con GWThttp://jooink.blogpsot.comsuper-source tagper InputStream & altre parti della JRE non disponibili in GWTbyte[] raster;UInt8ClampedArray image = ImageData.getData();private static native byte[] toArrayOfBytes(Uint8ClampedArray a) /*-{ return a;}-*/;Documented in JAPANESE !!!
  18. 18. How It Works cam WebRTC <video/> canvas ImageData Elemental/jsni WebGL + model video GWT(NyARToolkit) byte[] mv matrix
  19. 19. Demo http://www.jooink.com/experiments/DevFest2012
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×