building rich internetapplications with HTML5and WebGLtony parisimay 29, 2013
about meserial entrepreneurfounder, stealth startupconsulting architect and CTOco-creator, VRML and X3D web standardsautho...
“rich internet applications”term popularized by Adobefor building animated, media-rich, highly interactive web contentciph...
HTML5: the browser as platformgraphics APIs– Canvas, WebGL, SVGCSS3 – animations, transitions, transforms and filter effec...
how rich?http://www.tonyparisi.com5/29/2013ported in 5 days Unreal native C++ engine -> JavaScriptEmscripten + asm.js60FPS
WebGL:real-time 3D renderingthe new 3D API standardOpenGL ES™ in a browserJavaScript API bindingssupported in nearly all m...
how WebGL worksit’s a JavaScript drawing APIdraw to a canvas element using a special contextlow-level drawing – buffers, p...
a simple WebGL program1. create a <canvas> element2. obtain a drawing context3. initialize the viewport4. create one or mo...
buffers and typed arraysvar vertexBuffer;vertexBuffer = gl.createBuffer();gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer);var...
shadersvar vertexShaderSource =" attribute vec3 vertexPos;n" +" attribute vec2 texCoord;n" +" uniform mat4 modelViewMatrix...
drawingfunction draw(gl, obj) {// clear the background (with black)gl.clearColor(0.0, 0.0, 0.0, 1.0);gl.enable(gl.DEPTH_TE...
three.js:a JavaScript 3D enginethe most popular WebGL libraryrenderer = new THREE.WebGLRenderer( { canvas: canvas, antiali...
three.js flagship projectshttp://www.tonyparisi.com5/29/2013
animationrequestAnimationFrame()http://www.paulirish.com/2011/requestanimationframe-for-smart-animating/with JavaScript we...
pipeline toolsWebGL has no file format; no markuplanguage; no DOM. apps/libraries mustdefine their own formatsa lot of peo...
CSS3:advanced page effectsCSS transitionsCSS animationsCSS 3D transformsCSS filter effectsCSS custom filtershttp://www.ton...
CSS3 transitionsallow gradual changes to CSS properties over time#box img {transition: transform .5s ease-in;transform: tr...
CSS3 animationsallow fine control over animation behavior with key frames#bigcloud {animation: bobble 2s infinite;}@keyfra...
CSS3 3D transformstranslate, rotate, scale page elements with perspective.bk-list li {perspective: 1800px;}.bk-list li .bk...
CSS3 filter effectsapply post-processing effects to image elementssource: http://html5-demos.appspot.com/static/css/filter...
CSS3 custom filters.shader {-webkit-filter: custom(url(shaders/crumple.vs) mix(url(shaders/crumple.fs)multiply source-atop...
CSS3 vertex shadervoid main() {// Compute displacementfloat time = 10.0 * quadraticInOut( amount );vec2 surfaceCoordinate ...
CSS3 fragment shader// Uniform values from CSSuniform float amount;// Varyings passed in from vertex shadervarying vec2 v_...
CSS3 supportCSS3 Transitionssupported since• Safari 3.2: 13/11/2008• Firefox 4.0: Late 2010• Chrome 1.0: 02/09/2008• Opera...
fun with three.js and CSS3http://mrdoob.github.io/three.js/examples/css3d_periodictable.htmlhttp://www.tonyparisi.com5/29/...
language innovationsasm.jsoptimizable, low-level subset of JavaScript; strictly typed; improvesperformance (2x native vs. ...
always bet on JS5/29/2013http://www.tonyparisi.comFrom: Brendan Eich’s The State of JavaScripthttp://brendaneich.github.co...
resourcesWebGL specification http://www.khronos.org/registry/webgl/specs/latest/Learning WebGL http://www.learningwebgl.co...
stay in touchcontact informationtparisi@gmail.comskype: auradeluxehttp://twitter.com/auradeluxe http://www.tonyparisi.com/...
Upcoming SlideShare
Loading in …5
×

Building Rich Internet Applications with HTML5 and WebGL

3,152 views
3,013 views

Published on

Slides from my talk at O'Reilly Fluent 2013

1 Comment
6 Likes
Statistics
Notes
  • An Interesting and informative presentation that could help us gather the basics and get started off. HTML5 and Javascript are tools that help create very interesting apps. At Neev, we have been working on HTML5 and Javascript since long and have developed some cool apps. You can check them out at http://www.neevtech.com/client/.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
3,152
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
1
Likes
6
Embeds 0
No embeds

No notes for slide

Building Rich Internet Applications with HTML5 and WebGL

  1. 1. building rich internetapplications with HTML5and WebGLtony parisimay 29, 2013
  2. 2. about meserial entrepreneurfounder, stealth startupconsulting architect and CTOco-creator, VRML and X3D web standardsauthorcontact informationtparisi@gmail.comskype: auradeluxehttp://twitter.com/auradeluxe http://www.tonyparisi.com/http://www.learningwebgl.com/book source codehttps://github.com/tparisi/WebGLBookget the book!WebGL: Up and Runninghttp://shop.oreilly.com/product/0636920024729.dodiscount code: WEBGL13 (expires 6/7/13)print book ISBN: 9781449323578ebook ISBN: 9781449326500http://www.tonyparisi.com5/29/2013
  3. 3. “rich internet applications”term popularized by Adobefor building animated, media-rich, highly interactive web contentcipher for “you can’t do THAT in a browser without Flash player”vector graphics and animationlayering and compositingstreaming audio and videoprofessional-level scripting languagecomponent-based developmentTCP/IP socketshttp://www.tonyparisi.com5/29/2013
  4. 4. HTML5: the browser as platformgraphics APIs– Canvas, WebGL, SVGCSS3 – animations, transitions, transforms and filter effectshardware-accelerated compositingall elements blend seamlessly on pageJavaScript performance and language enhancementsand a whole lot more…first-class streaming media types - <audio>, <video>device inputlocation and mobile-inspired featuresdatabase, local storage, file systemnetworking – sockets and real-time connections (WebRTC)multi-threading (Workers)http://www.tonyparisi.com5/29/2013
  5. 5. how rich?http://www.tonyparisi.com5/29/2013ported in 5 days Unreal native C++ engine -> JavaScriptEmscripten + asm.js60FPS
  6. 6. WebGL:real-time 3D renderingthe new 3D API standardOpenGL ES™ in a browserJavaScript API bindingssupported in nearly all modern browserssupported on many devicesshipped since early 2011supported in• Safari, Firefox, Chrome, Opera• Internet Explorer (late 2013)• iOS – iAds only• Android – mobile Chrome• Blackberry, Tizen, Firefox OS• 500M+ seats -> 1Bhttp://www.tonyparisi.com5/29/2013…and it’sawesome.100,000 Stars Google Experiment
  7. 7. how WebGL worksit’s a JavaScript drawing APIdraw to a canvas element using a special contextlow-level drawing – buffers, primitives, textures and shadersaccelerated by graphics hardware (GPU)can draw 2D as well as 3D graphicsthere is no file format; no markup language; no DOM.http://www.tonyparisi.com5/29/2013
  8. 8. a simple WebGL program1. create a <canvas> element2. obtain a drawing context3. initialize the viewport4. create one or more buffers5. create one or more matrices6. create one or more shaders7. initialize the shaders8. draw one or more primitiveshttp://www.tonyparisi.com5/29/2013
  9. 9. buffers and typed arraysvar vertexBuffer;vertexBuffer = gl.createBuffer();gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer);var verts = [// Front face-1.0, -1.0, 1.0,1.0, -1.0, 1.0,1.0, 1.0, 1.0,-1.0, 1.0, 1.0,// Back face-1.0, -1.0, -1.0,-1.0, 1.0, -1.0,1.0, 1.0, -1.0,1.0, -1.0, -1.0,…];gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(verts), gl.STATIC_DRAW);WebGL drawing functionsuse buffers of datanew low-level data typestores arrays of floatsand ints compactlyhttp://www.tonyparisi.com5/29/2013
  10. 10. shadersvar vertexShaderSource =" attribute vec3 vertexPos;n" +" attribute vec2 texCoord;n" +" uniform mat4 modelViewMatrix;n" +" uniform mat4 projectionMatrix;n" +" varying vec2 vTexCoord;n" +" void main(void) {n" +" // Return the transformed and projected vertex valuen" +" gl_Position = projectionMatrix * modelViewMatrix * n" +" vec4(vertexPos, 1.0);n" +" // Output the texture coordinate in vTexCoordn" +" vTexCoord = texCoord;n" +" }n";var fragmentShaderSource =" precision mediump float;n" +" varying vec2 vTexCoord;n" +" uniform sampler2D uSampler;n" +" void main(void) {n" +" // Return the pixel color: always output whiten" +" gl_FragColor = texture2D(uSampler, vec2(vTexCoord.s, vTexCoord.t));n" +"}n";the vertex shadertransforms model-spacepositions into screenspacethe fragment shaderoutputs a color value foreach pixelhttp://www.tonyparisi.com5/29/2013
  11. 11. drawingfunction draw(gl, obj) {// clear the background (with black)gl.clearColor(0.0, 0.0, 0.0, 1.0);gl.enable(gl.DEPTH_TEST);gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);// set the shader to usegl.useProgram(shaderProgram);// connect up the shader parameters: vertex position, texture coordinate,// projection/model matrices and texture// set up the buffersgl.bindBuffer(gl.ARRAY_BUFFER, obj.buffer);gl.vertexAttribPointer(shaderVertexPositionAttribute, obj.vertSize, gl.FLOAT, false, 0, 0);gl.bindBuffer(gl.ARRAY_BUFFER, obj.texCoordBuffer);gl.vertexAttribPointer(shaderTexCoordAttribute, obj.texCoordSize, gl.FLOAT, false, 0, 0);gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, obj.indices);gl.uniformMatrix4fv(shaderProjectionMatrixUniform, false, projectionMatrix);gl.uniformMatrix4fv(shaderModelViewMatrixUniform, false, modelViewMatrix);gl.activeTexture(gl.TEXTURE0);gl.bindTexture(gl.TEXTURE_2D, webGLTexture);gl.uniform1i(shaderSamplerUniform, 0);// draw the objectgl.drawElements(obj.primtype, obj.nIndices, gl.UNSIGNED_SHORT, 0);}clear the canvasset the shaderset up buffers forvertices andtexturecoordinatespass transformand projectionmatricesto the shaderset the texture and pass tothe shaderdraw the objecthttp://www.tonyparisi.com5/29/2013
  12. 12. three.js:a JavaScript 3D enginethe most popular WebGL libraryrenderer = new THREE.WebGLRenderer( { canvas: canvas, antialias: true } );scene = new THREE.Scene();camera = new THREE.PerspectiveCamera( 45, canvas.width /canvas.height, 1, 4000 );scene.add(camera);var light = new THREE.DirectionalLight( 0xffffff, 1.5);scene.add( light );var mapUrl ="../images/webgl-logo-256.jpg“;var map = THREE.ImageUtils.loadTexture(mapUrl );var material = new THREE.MeshPhongMaterial({ map: map });var geometry = new THREE.CubeGeometry(2, 2, 2);cube = new THREE.Mesh(geometry, material);scene.add( cube );https://github.com/mrdoob/three.js/representsWebGL at ahigh level usingstandard3D graphicsconceptscan render toWebGL,2D canvas, SVGand CSS3http://www.tonyparisi.com5/29/2013
  13. 13. three.js flagship projectshttp://www.tonyparisi.com5/29/2013
  14. 14. animationrequestAnimationFrame()http://www.paulirish.com/2011/requestanimationframe-for-smart-animating/with JavaScript we can animate anything: materials,lights, textures…. shadersthree.js has support for key frames, morphs and skinsTween.js – simple tweening libraryhttps://github.com/sole/tween.js/var tween = new TWEEN.Tween( { x: 50, y: 0 } ).to( { x: 400 }, 2000 ).easing( TWEEN.Easing.Elastic.InOut ).start();function animate() {requestAnimationFrame( animate );TWEEN.update();}create andstart tweencall TWEEN.update()in your run loophttp://www.tonyparisi.com5/29/2013
  15. 15. pipeline toolsWebGL has no file format; no markuplanguage; no DOM. apps/libraries mustdefine their own formatsa lot of people are still creating content bywriting codethree.js loads several format types, but theloaders are incomplete and buggy3D format standardsCOLLADA – XMLhttp://www.khronos.org/files/collada_spec_1_4.pdfglTF – compact JSON + binaryhttps://github.com/KhronosGroup/glTFart paths from 3ds Max, Maya, Blender,Sketchup5/29/2013http://www.tonyparisi.comMaya file exported as COLLADA,loaded with three.js
  16. 16. CSS3:advanced page effectsCSS transitionsCSS animationsCSS 3D transformsCSS filter effectsCSS custom filtershttp://www.tonyparisi.com5/29/2013
  17. 17. CSS3 transitionsallow gradual changes to CSS properties over time#box img {transition: transform .5s ease-in;transform: translate3d(0, -350px, 0);}#box img:hover {transition: transform .5s ease-in;transform: translate3d(0, 0px, 0);cursor: pointer;}transition: all .5s ease-in -.1s;transition-property: all;transition-duration: .5s;transition-timing-function: ease-in;transition-delay: .1s;source: http://www.kirupa.com/html5/all_about_css_transitions.htmshortcut forspecify property totransition, duration oftransition, easingfunctionhttp://www.tonyparisi.com5/29/2013
  18. 18. CSS3 animationsallow fine control over animation behavior with key frames#bigcloud {animation: bobble 2s infinite;}@keyframes bobble {0% {transform: translate3d(50px, 40px, 0px);animation-timing-function: ease-in;}50% {transform: translate3d(50px, 50px, 0px);animation-timing-function: ease-out;}100% {transform: translate3d(50px, 40px, 0px);}}keys are used to interpolatetransform valuescontrol timing withease in/out functionseach key frame contains a transformsource: http://www.kirupa.com/html5/all_about_css_animations.htmhttp://www.tonyparisi.com5/29/2013
  19. 19. CSS3 3D transformstranslate, rotate, scale page elements with perspective.bk-list li {perspective: 1800px;}.bk-list li .bk-front {transform-style: preserve-3d;transform-origin: 0% 50%;transform: translate3d(0,0,20px);}.bk-list li .bk-book.bk-bookdefault:hover {transform: rotate3d(0,1,0,35deg);}sourcehttp://tympanus.net/codrops/2013/01/08/3d-book-showcase/browser will render element in 3D perspectiveadd origin totranslationapply to childelementsapply 35 degree rotation about Y axishttp://www.tonyparisi.com5/29/2013
  20. 20. CSS3 filter effectsapply post-processing effects to image elementssource: http://html5-demos.appspot.com/static/css/filters/index.htmlimg {-webkit-filter: sepia(100%);}img {-webkit-filter: grayscale(100%);}img {-webkit-filter: blur(2px);}img {-webkit-filter: brightness(33%);}img {-webkit-filter: contrast(67%);}http://www.tonyparisi.com5/29/2013
  21. 21. CSS3 custom filters.shader {-webkit-filter: custom(url(shaders/crumple.vs) mix(url(shaders/crumple.fs)multiply source-atop), 50 50, amount 0, strength 0.2, lightIntensity 1.05, transformrotateX(0deg) translateZ(0px) );-webkit-transition: -webkit-filter linear 1.5s;box-shadow: 0 0 2em #111;}.shader:hover {-webkit-filter: custom(url(shaders/crumple.vs) mix(url(shaders/crumple.fs)multiply source-atop), 50 50, amount 1, strength 0.2, lightIntensity 1.05, transformrotateX(0deg) translateZ(0px) );}source: http://alteredqualia.com/css-shaders/crumple.htmlhttp://www.tonyparisi.com5/29/2013Crumple shader byBranislav Ulicny (AlteredQualia)http://alteredqualia.com/css-shaders/crumple.html
  22. 22. CSS3 vertex shadervoid main() {// Compute displacementfloat time = 10.0 * quadraticInOut( amount );vec2 surfaceCoordinate = 0.025 * a_texCoord;float n = surface( surfaceCoordinate, time );float curve = strength * n;vec4 pos = a_position;pos.z = amount * ( 0.5 * strength - curve );// Compute normalconst float e = 0.001;float nx = surface( surfaceCoordinate + vec2( e, 0.0 ), time );float ny = surface( surfaceCoordinate + vec2( 0.0, e ), time );vec3 normal = normalize( vec3( n - nx, n - ny, 1.0 - strength * 1.25 ) );// Compute lighting (directional light)vec3 lightPosition = normalize( vec3( 0.0, 0.5, 1.0 ) );float lightWeight = lightIntensity * max( dot( normal, lightPosition ), 0.0 );// Set vertex positiongl_Position = u_projectionMatrix * perspective( 0.9 ) * transform * pos;v_uv = a_texCoord;v_height = n;v_light = lightWeight;browser predefinesa_texCoord anda_position, passes intoshadershaderoutputs newvertexposition,passeslightinginformation tofragmentshaderhttp://www.tonyparisi.com5/29/2013
  23. 23. CSS3 fragment shader// Uniform values from CSSuniform float amount;// Varyings passed in from vertex shadervarying vec2 v_uv;varying float v_height;varying float v_light;void main() {const float a = 1.0;float r, g, b;// Light variantfloat n = v_light;float v = mix( 1.0, n * n, amount );r = g = b = sqrt( v );// Set color matrixcss_ColorMatrix = mat4( r, 0.0, 0.0, 0.0,0.0, g, 0.0, 0.0,0.0, 0.0, b, 0.0,0.0, 0.0, 0.0, a );}Fragmentshaderoutputs blendvalue basedon lightcalculated invertex shaderand user-suppliedamounthttp://www.tonyparisi.com5/29/2013
  24. 24. CSS3 supportCSS3 Transitionssupported since• Safari 3.2: 13/11/2008• Firefox 4.0: Late 2010• Chrome 1.0: 02/09/2008• Opera 10.5: 02/03/2010• Internet Explorer 10: 09/2011CSS3 Animationssupported since• Safari 4.0: 11/06/2008• Chrome 1.0: 02/09/2008• Firefox 5: 20/04/2011• IE 10: 09/2011• Opera: 03/2012CSS 3D Transformssupported since• Safari 4.0: 11/06/2008• Chrome: 28/08/2010• IE 10: 09/2011• Firefox: 27/10/2011source: http://css3.bradshawenterprises.com/CSS3 Filterssupported since• Safari 6.0• Chrome: 18.0CSS3 Custom Filterssupported only in Chromehttp://www.tonyparisi.com5/29/2013
  25. 25. fun with three.js and CSS3http://mrdoob.github.io/three.js/examples/css3d_periodictable.htmlhttp://www.tonyparisi.com5/29/2013
  26. 26. language innovationsasm.jsoptimizable, low-level subset of JavaScript; strictly typed; improvesperformance (2x native vs. 3-10x) runs in FF and Chrome nightlieshttp://asmjs.org/EmscriptenLLVM-to-JavaScript compiler translates C++ native code toJavaScript; can output to asm.jshttps://github.com/kripken/emscripten/wikinew Web languages - Dart, TypeScriptoptionally strongly typed; class-basedhttps://www.dartlang.org/http://www.typescriptlang.org/tackling performance and memory challengeshttp://www.tonyparisi.com5/29/2013
  27. 27. always bet on JS5/29/2013http://www.tonyparisi.comFrom: Brendan Eich’s The State of JavaScripthttp://brendaneich.github.com/Strange-Loop-2012/#/So, Ive thought for a longtime ... if I could take aclean sheet of paper andwrite [a new language] thatretains all the goodness of[JavaScript] ... I would nothave come up with anythinglike Dart.-- Douglas Crockford
  28. 28. resourcesWebGL specification http://www.khronos.org/registry/webgl/specs/latest/Learning WebGL http://www.learningwebgl.com/three.js https://github.com/mrdoob/three.js/requestAnimationFramehttp://www.paulirish.com/2011/requestanimationframe-for-smart-animating/CSS specificationshttp://www.w3.org/TR/css3-animations/http://www.w3.org/TR/css3-transitions/http://www.w3.org/TR/css3-transforms/http://www.w3.org/TR/filter-effects/CSS animation demos http://www.creativebloq.com/css3/animation-with-css3-712437CSS 3D transforms explained http://desandro.github.io/3dtransforms/Adobe filter lab - playing with filtereffectshttp://html.adobe.com/webplatform/graphics/customfilters/cssfilterlab/Alan Greenblatt’s blog – CSS filter effects http://blattchat.com/http://www.tonyparisi.com5/29/2013
  29. 29. stay in touchcontact informationtparisi@gmail.comskype: auradeluxehttp://twitter.com/auradeluxe http://www.tonyparisi.com/http://www.learningwebgl.com/get the book!WebGL: Up and Runninghttp://shop.oreilly.com/product/0636920024729.dodiscount code: WEBGL13 (expires 6/7/13)print book ISBN: 9781449323578ebook ISBN: 9781449326500http://www.tonyparisi.com5/29/2013book source codehttps://github.com/tparisi/WebGLBook

×