Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Building Rich Internet Applications with HTML5 and WebGL


Published on

Slides from my talk at O'Reilly Fluent 2013

  • An Interesting and informative presentation that could help us gather the basics and get started off. HTML5 and Javascript are tools that help create very interesting apps. At Neev, we have been working on HTML5 and Javascript since long and have developed some cool apps. You can check them out at
    Are you sure you want to  Yes  No
    Your message goes here

Building Rich Internet Applications with HTML5 and WebGL

  1. 1. building rich internetapplications with HTML5and WebGLtony parisimay 29, 2013
  2. 2. about meserial entrepreneurfounder, stealth startupconsulting architect and CTOco-creator, VRML and X3D web standardsauthorcontact informationtparisi@gmail.comskype: auradeluxe source code the book!WebGL: Up and Running code: WEBGL13 (expires 6/7/13)print book ISBN: 9781449323578ebook ISBN: 9781449326500http://www.tonyparisi.com5/29/2013
  3. 3. “rich internet applications”term popularized by Adobefor building animated, media-rich, highly interactive web contentcipher for “you can’t do THAT in a browser without Flash player”vector graphics and animationlayering and compositingstreaming audio and videoprofessional-level scripting languagecomponent-based developmentTCP/IP socketshttp://www.tonyparisi.com5/29/2013
  4. 4. HTML5: the browser as platformgraphics APIs– Canvas, WebGL, SVGCSS3 – animations, transitions, transforms and filter effectshardware-accelerated compositingall elements blend seamlessly on pageJavaScript performance and language enhancementsand a whole lot more…first-class streaming media types - <audio>, <video>device inputlocation and mobile-inspired featuresdatabase, local storage, file systemnetworking – sockets and real-time connections (WebRTC)multi-threading (Workers)http://www.tonyparisi.com5/29/2013
  5. 5. how rich?http://www.tonyparisi.com5/29/2013ported in 5 days Unreal native C++ engine -> JavaScriptEmscripten + asm.js60FPS
  6. 6. WebGL:real-time 3D renderingthe new 3D API standardOpenGL ES™ in a browserJavaScript API bindingssupported in nearly all modern browserssupported on many devicesshipped since early 2011supported in• Safari, Firefox, Chrome, Opera• Internet Explorer (late 2013)• iOS – iAds only• Android – mobile Chrome• Blackberry, Tizen, Firefox OS• 500M+ seats -> 1Bhttp://www.tonyparisi.com5/29/2013…and it’sawesome.100,000 Stars Google Experiment
  7. 7. how WebGL worksit’s a JavaScript drawing APIdraw to a canvas element using a special contextlow-level drawing – buffers, primitives, textures and shadersaccelerated by graphics hardware (GPU)can draw 2D as well as 3D graphicsthere is no file format; no markup language; no DOM.http://www.tonyparisi.com5/29/2013
  8. 8. a simple WebGL program1. create a <canvas> element2. obtain a drawing context3. initialize the viewport4. create one or more buffers5. create one or more matrices6. create one or more shaders7. initialize the shaders8. draw one or more primitiveshttp://www.tonyparisi.com5/29/2013
  9. 9. buffers and typed arraysvar vertexBuffer;vertexBuffer = gl.createBuffer();gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer);var verts = [// Front face-1.0, -1.0, 1.0,1.0, -1.0, 1.0,1.0, 1.0, 1.0,-1.0, 1.0, 1.0,// Back face-1.0, -1.0, -1.0,-1.0, 1.0, -1.0,1.0, 1.0, -1.0,1.0, -1.0, -1.0,…];gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(verts), gl.STATIC_DRAW);WebGL drawing functionsuse buffers of datanew low-level data typestores arrays of floatsand ints compactlyhttp://www.tonyparisi.com5/29/2013
  10. 10. shadersvar vertexShaderSource =" attribute vec3 vertexPos;n" +" attribute vec2 texCoord;n" +" uniform mat4 modelViewMatrix;n" +" uniform mat4 projectionMatrix;n" +" varying vec2 vTexCoord;n" +" void main(void) {n" +" // Return the transformed and projected vertex valuen" +" gl_Position = projectionMatrix * modelViewMatrix * n" +" vec4(vertexPos, 1.0);n" +" // Output the texture coordinate in vTexCoordn" +" vTexCoord = texCoord;n" +" }n";var fragmentShaderSource =" precision mediump float;n" +" varying vec2 vTexCoord;n" +" uniform sampler2D uSampler;n" +" void main(void) {n" +" // Return the pixel color: always output whiten" +" gl_FragColor = texture2D(uSampler, vec2(vTexCoord.s, vTexCoord.t));n" +"}n";the vertex shadertransforms model-spacepositions into screenspacethe fragment shaderoutputs a color value foreach pixelhttp://www.tonyparisi.com5/29/2013
  11. 11. drawingfunction draw(gl, obj) {// clear the background (with black)gl.clearColor(0.0, 0.0, 0.0, 1.0);gl.enable(gl.DEPTH_TEST);gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);// set the shader to usegl.useProgram(shaderProgram);// connect up the shader parameters: vertex position, texture coordinate,// projection/model matrices and texture// set up the buffersgl.bindBuffer(gl.ARRAY_BUFFER, obj.buffer);gl.vertexAttribPointer(shaderVertexPositionAttribute, obj.vertSize, gl.FLOAT, false, 0, 0);gl.bindBuffer(gl.ARRAY_BUFFER, obj.texCoordBuffer);gl.vertexAttribPointer(shaderTexCoordAttribute, obj.texCoordSize, gl.FLOAT, false, 0, 0);gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, obj.indices);gl.uniformMatrix4fv(shaderProjectionMatrixUniform, false, projectionMatrix);gl.uniformMatrix4fv(shaderModelViewMatrixUniform, false, modelViewMatrix);gl.activeTexture(gl.TEXTURE0);gl.bindTexture(gl.TEXTURE_2D, webGLTexture);gl.uniform1i(shaderSamplerUniform, 0);// draw the objectgl.drawElements(obj.primtype, obj.nIndices, gl.UNSIGNED_SHORT, 0);}clear the canvasset the shaderset up buffers forvertices andtexturecoordinatespass transformand projectionmatricesto the shaderset the texture and pass tothe shaderdraw the objecthttp://www.tonyparisi.com5/29/2013
  12. 12. three.js:a JavaScript 3D enginethe most popular WebGL libraryrenderer = new THREE.WebGLRenderer( { canvas: canvas, antialias: true } );scene = new THREE.Scene();camera = new THREE.PerspectiveCamera( 45, canvas.width /canvas.height, 1, 4000 );scene.add(camera);var light = new THREE.DirectionalLight( 0xffffff, 1.5);scene.add( light );var mapUrl ="../images/webgl-logo-256.jpg“;var map = THREE.ImageUtils.loadTexture(mapUrl );var material = new THREE.MeshPhongMaterial({ map: map });var geometry = new THREE.CubeGeometry(2, 2, 2);cube = new THREE.Mesh(geometry, material);scene.add( cube ); at ahigh level usingstandard3D graphicsconceptscan render toWebGL,2D canvas, SVGand CSS3http://www.tonyparisi.com5/29/2013
  13. 13. three.js flagship projectshttp://www.tonyparisi.com5/29/2013
  14. 14. animationrequestAnimationFrame() JavaScript we can animate anything: materials,lights, textures…. shadersthree.js has support for key frames, morphs and skinsTween.js – simple tweening library tween = new TWEEN.Tween( { x: 50, y: 0 } ).to( { x: 400 }, 2000 ).easing( TWEEN.Easing.Elastic.InOut ).start();function animate() {requestAnimationFrame( animate );TWEEN.update();}create andstart tweencall TWEEN.update()in your run loophttp://www.tonyparisi.com5/29/2013
  15. 15. pipeline toolsWebGL has no file format; no markuplanguage; no DOM. apps/libraries mustdefine their own formatsa lot of people are still creating content bywriting codethree.js loads several format types, but theloaders are incomplete and buggy3D format standardsCOLLADA – XML – compact JSON + binary paths from 3ds Max, Maya, Blender,Sketchup5/29/2013http://www.tonyparisi.comMaya file exported as COLLADA,loaded with three.js
  16. 16. CSS3:advanced page effectsCSS transitionsCSS animationsCSS 3D transformsCSS filter effectsCSS custom filtershttp://www.tonyparisi.com5/29/2013
  17. 17. CSS3 transitionsallow gradual changes to CSS properties over time#box img {transition: transform .5s ease-in;transform: translate3d(0, -350px, 0);}#box img:hover {transition: transform .5s ease-in;transform: translate3d(0, 0px, 0);cursor: pointer;}transition: all .5s ease-in -.1s;transition-property: all;transition-duration: .5s;transition-timing-function: ease-in;transition-delay: .1s;source: forspecify property totransition, duration oftransition, easingfunctionhttp://www.tonyparisi.com5/29/2013
  18. 18. CSS3 animationsallow fine control over animation behavior with key frames#bigcloud {animation: bobble 2s infinite;}@keyframes bobble {0% {transform: translate3d(50px, 40px, 0px);animation-timing-function: ease-in;}50% {transform: translate3d(50px, 50px, 0px);animation-timing-function: ease-out;}100% {transform: translate3d(50px, 40px, 0px);}}keys are used to interpolatetransform valuescontrol timing withease in/out functionseach key frame contains a transformsource:
  19. 19. CSS3 3D transformstranslate, rotate, scale page elements with perspective.bk-list li {perspective: 1800px;}.bk-list li .bk-front {transform-style: preserve-3d;transform-origin: 0% 50%;transform: translate3d(0,0,20px);}.bk-list li .bk-book.bk-bookdefault:hover {transform: rotate3d(0,1,0,35deg);}source will render element in 3D perspectiveadd origin totranslationapply to childelementsapply 35 degree rotation about Y axishttp://www.tonyparisi.com5/29/2013
  20. 20. CSS3 filter effectsapply post-processing effects to image elementssource: {-webkit-filter: sepia(100%);}img {-webkit-filter: grayscale(100%);}img {-webkit-filter: blur(2px);}img {-webkit-filter: brightness(33%);}img {-webkit-filter: contrast(67%);}http://www.tonyparisi.com5/29/2013
  21. 21. CSS3 custom filters.shader {-webkit-filter: custom(url(shaders/crumple.vs) mix(url(shaders/crumple.fs)multiply source-atop), 50 50, amount 0, strength 0.2, lightIntensity 1.05, transformrotateX(0deg) translateZ(0px) );-webkit-transition: -webkit-filter linear 1.5s;box-shadow: 0 0 2em #111;}.shader:hover {-webkit-filter: custom(url(shaders/crumple.vs) mix(url(shaders/crumple.fs)multiply source-atop), 50 50, amount 1, strength 0.2, lightIntensity 1.05, transformrotateX(0deg) translateZ(0px) );}source: shader byBranislav Ulicny (AlteredQualia)
  22. 22. CSS3 vertex shadervoid main() {// Compute displacementfloat time = 10.0 * quadraticInOut( amount );vec2 surfaceCoordinate = 0.025 * a_texCoord;float n = surface( surfaceCoordinate, time );float curve = strength * n;vec4 pos = a_position;pos.z = amount * ( 0.5 * strength - curve );// Compute normalconst float e = 0.001;float nx = surface( surfaceCoordinate + vec2( e, 0.0 ), time );float ny = surface( surfaceCoordinate + vec2( 0.0, e ), time );vec3 normal = normalize( vec3( n - nx, n - ny, 1.0 - strength * 1.25 ) );// Compute lighting (directional light)vec3 lightPosition = normalize( vec3( 0.0, 0.5, 1.0 ) );float lightWeight = lightIntensity * max( dot( normal, lightPosition ), 0.0 );// Set vertex positiongl_Position = u_projectionMatrix * perspective( 0.9 ) * transform * pos;v_uv = a_texCoord;v_height = n;v_light = lightWeight;browser predefinesa_texCoord anda_position, passes intoshadershaderoutputs newvertexposition,passeslightinginformation tofragmentshaderhttp://www.tonyparisi.com5/29/2013
  23. 23. CSS3 fragment shader// Uniform values from CSSuniform float amount;// Varyings passed in from vertex shadervarying vec2 v_uv;varying float v_height;varying float v_light;void main() {const float a = 1.0;float r, g, b;// Light variantfloat n = v_light;float v = mix( 1.0, n * n, amount );r = g = b = sqrt( v );// Set color matrixcss_ColorMatrix = mat4( r, 0.0, 0.0, 0.0,0.0, g, 0.0, 0.0,0.0, 0.0, b, 0.0,0.0, 0.0, 0.0, a );}Fragmentshaderoutputs blendvalue basedon lightcalculated invertex shaderand user-suppliedamounthttp://www.tonyparisi.com5/29/2013
  24. 24. CSS3 supportCSS3 Transitionssupported since• Safari 3.2: 13/11/2008• Firefox 4.0: Late 2010• Chrome 1.0: 02/09/2008• Opera 10.5: 02/03/2010• Internet Explorer 10: 09/2011CSS3 Animationssupported since• Safari 4.0: 11/06/2008• Chrome 1.0: 02/09/2008• Firefox 5: 20/04/2011• IE 10: 09/2011• Opera: 03/2012CSS 3D Transformssupported since• Safari 4.0: 11/06/2008• Chrome: 28/08/2010• IE 10: 09/2011• Firefox: 27/10/2011source: Filterssupported since• Safari 6.0• Chrome: 18.0CSS3 Custom Filterssupported only in Chromehttp://www.tonyparisi.com5/29/2013
  25. 25. fun with three.js and CSS3
  26. 26. language innovationsasm.jsoptimizable, low-level subset of JavaScript; strictly typed; improvesperformance (2x native vs. 3-10x) runs in FF and Chrome nightlies compiler translates C++ native code toJavaScript; can output to asm.js Web languages - Dart, TypeScriptoptionally strongly typed; class-based performance and memory challengeshttp://www.tonyparisi.com5/29/2013
  27. 27. always bet on JS5/29/2013http://www.tonyparisi.comFrom: Brendan Eich’s The State of JavaScript, Ive thought for a longtime ... if I could take aclean sheet of paper andwrite [a new language] thatretains all the goodness of[JavaScript] ... I would nothave come up with anythinglike Dart.-- Douglas Crockford
  28. 28. resourcesWebGL specification WebGL specifications animation demos 3D transforms explained filter lab - playing with filtereffects Greenblatt’s blog – CSS filter effects
  29. 29. stay in touchcontact informationtparisi@gmail.comskype: auradeluxe the book!WebGL: Up and Running code: WEBGL13 (expires 6/7/13)print book ISBN: 9781449323578ebook ISBN: 9781449326500http://www.tonyparisi.com5/29/2013book source code