Successfully reported this slideshow.

Developing Web Graphics with WebGL

7,249 views

Published on

Slides From My Presentation at HTML5 Dev Conf

Published in: Technology, Art & Photos
  • Be the first to comment

Developing Web Graphics with WebGL

  1. 1. developing web graphics with WebGL tony parisi October 22, 2013
  2. 2. about me serial entrepreneur founder, stealth startup consulting architect and CTO co-creator, VRML and X3D web standards co-designer, glTF author, speaker instructor contact information tparisi@gmail.com skype: auradeluxe http://twitter.com/auradeluxe p://www.tonyparisi.com/ http://www.learningwebgl.com/ book source code https://github.com/tparisi/WebGLBook https://github.com/tparisi/Programming3DApplications htt get the books! WebGL: Up and Running http://www.amazon.com/dp/144932357X Programming 3D Applications with HTML and WebGL http://www.amazon.com/Programming-ApplicationsHTML5-WebGL-Visualization/dp/1449362966 http://www.tonyparisi.com 10/22/2 013
  3. 3. RIP: 1995-2013 Image: Eric Dye http://www.tonyparisi.com 10/22/2 013
  4. 4. long live… http://www.tonyparisi.com 10/22/2 013
  5. 5. WebGL: real-time 3D rendering the 3D API standard OpenGL ES™ in a browser JavaScript API bindings supported in all modern browsers shipped since early 2011 supported in • • • • • desktop Safari, Firefox, Chrome, Opera Internet Explorer (late 2013) iOS mobile Safari – iAds only Android – mobile Chrome, mobile Firefox Blackberry, Tizen, Firefox OS • Surface (Windows 8.1) • • Kindle Fire HDX 500M+ seats -> 1B http://www.tonyparisi.com 10/22/2 013
  6. 6. science and education 100,000 Stars Google Experiment http://workshop.chromeexperiments.com/stars/ http://www.tonyparisi.com 10/22/2 013
  7. 7. advertising and media collaboration with Rei Inamoto and AKQA http://makeourmark.levi.com/project-overview-whatmovesyou.html developed by Tony Parisi and Simo Santavirta http://www.simppa.fi/ http://www.tonyparisi.com 10/22/2 013
  8. 8. data visualization http://www.tonyparisi.com 10/22/2 013
  9. 9. page graphics Steve Wittens http://acko.net/blog/making-mathbox/ http://www.tonyparisi.com 10/22/2 013
  10. 10. games and virtual environments 60FPS ported in 5 days Unreal native C++ engine -> JavaScriptEmscripten + asm.js http://www.tonyparisi.com 10/22/2 013
  11. 11. products and e-commerce product concept piece – Vizi - TC Chang/T. Parisi http://vizi.gl/engine/tests/futurgo.html http://www.tonyparisi.com 10/22/2 013
  12. 12. how WebGL works it’s a JavaScript drawing API draw to a canvas element using a special context (“webgl”) low-level drawing – buffers, primitives, textures and shaders accelerated by graphics hardware (GPU) can draw 2D as well as 3D graphics integrates seamlessly with other page content there is no file format; no markup language; no DOM. libraries and frameworks are key to fast ramp up and productive development http://www.tonyparisi.com 10/22/2 013
  13. 13. a simple WebGL program 1. create a <canvas> element 2. obtain a drawing context 3. initialize the viewport 4. create one or more buffers 5. create one or more matrices 6. create one or more shaders 7. initialize the shaders 8. draw one or more primitives http://www.tonyparisi.com 10/22/2 013
  14. 14. create the canvas, context and viewport function initWebGL(canvas) { var gl = null; var msg = "Your browser does not support WebGL, " + "or it is not enabled by default."; try { gl = canvas.getContext(“webgl"); } catch (e) { msg = "Error creating WebGL Context!: " + e.toString(); } detect WebGL if (!gl) { alert(msg); throw new Error(msg); } return gl; } function initViewport(gl, canvas) { gl.viewport(0, 0, canvas.width, canvas.height); } set WebGL drawing region http://www.tonyparisi.com 10/22/2 013
  15. 15. buffers and typed arrays var vertexBuffer; vertexBuffer = gl.createBuffer(); gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer); var verts = [ // Front face -1.0, -1.0, 1.0, 1.0, -1.0, 1.0, 1.0, 1.0, 1.0, -1.0, 1.0, 1.0, // Back face -1.0, -1.0, -1.0, -1.0, 1.0, -1.0, 1.0, 1.0, -1.0, 1.0, -1.0, -1.0, … WebGL drawing functions use buffers of data new low-level data type stores arrays of floats and ints compactly ]; gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(verts), gl.STATIC_DRAW); http://www.tonyparisi.com 10/22/2 013
  16. 16. shaders var vertexShaderSource = " " " " " " " " " " " " the vertex shader attribute vec3 vertexPos;n" + attribute vec2 texCoord;n" + transforms model-space uniform mat4 modelViewMatrix;n" + positions into screen uniform mat4 projectionMatrix;n" + space varying vec2 vTexCoord;n" + void main(void) {n" + // Return the transformed and projected vertex valuen" + gl_Position = projectionMatrix * modelViewMatrix * n" + vec4(vertexPos, 1.0);n" + // Output the texture coordinate in vTexCoordn" + vTexCoord = texCoord;n" + }n"; the fragment shader var fragmentShaderSource = " precision mediump float;n" + outputs a color value " varying vec2 vTexCoord;n" + for each pixel " uniform sampler2D uSampler;n" + " void main(void) {n" + " // Return the pixel color: always output whiten" + " gl_FragColor = texture2D(uSampler, vec2(vTexCoord.s, vTexCoord.t));n" + "}n"; http://www.tonyparisi.com 10/22/2 013
  17. 17. drawing function draw(gl, obj) { // clear the background (with black) gl.clearColor(0.0, 0.0, 0.0, 1.0); gl.enable(gl.DEPTH_TEST); gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT); clear the canvas // set the shader to use gl.useProgram(shaderProgram); set the shader // connect up the shader parameters: vertex position, texture coordinate, // projection/model matrices and texture // set up the buffers gl.bindBuffer(gl.ARRAY_BUFFER, obj.buffer); gl.vertexAttribPointer(shaderVertexPositionAttribute, obj.vertSize, gl.FLOAT, false, 0, 0); gl.bindBuffer(gl.ARRAY_BUFFER, obj.texCoordBuffer); gl.vertexAttribPointer(shaderTexCoordAttribute, obj.texCoordSize, gl.FLOAT, false, 0, 0); gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, obj.indices); gl.uniformMatrix4fv(shaderProjectionMatrixUniform, false, projectionMatrix); gl.uniformMatrix4fv(shaderModelViewMatrixUniform, false, modelViewMatrix); gl.activeTexture(gl.TEXTURE0); gl.bindTexture(gl.TEXTURE_2D, webGLTexture); gl.uniform1i(shaderSamplerUniform, 0); // draw the object gl.drawElements(obj.primtype, obj.nIndices, gl.UNSIGNED_SHORT, 0); } set up buffers for vertices and texture coordinates pass transform and projection matrices to the shader set the texture and pass to the shader draw the object http://www.tonyparisi.com 10/22/2 013
  18. 18. a WebGL client-side stack rendering – Three.js library animation – Three.js, Tween.js libraries application functionality – Vizi framework setup/teardown interaction – picking, rollovers, rotating models behaviors run loop and event dispatching content creation pipeline 3D tools e.g. Autodesk Maya COLLADA2GLTF converter Vizi web-based previewer http://www.tonyparisi.com 10/22/2 013
  19. 19. three.js: a JavaScript 3D engine https://github.com/mrdoob/three.js/ the most popular WebGL library renderer = new THREE.WebGLRenderer( { canvas: canvas, antialias: true } ); scene = new THREE.Scene(); camera = new THREE.PerspectiveCamera( 45, canvas.width / canvas.height, 1, 4000 ); scene.add(camera); var light = new THREE.DirectionalLight( 0xffffff, 1.5); scene.add( light ); var mapUrl ="../images/webgl-logo-256.jpg“; var map = THREE.ImageUtils.loadTexture(mapUrl ); var material = new THREE.MeshPhongMaterial({ map: map }); var geometry = new THREE.CubeGeometry(2, 2, 2); cube = new THREE.Mesh(geometry, material); scene.add( cube ); can render to WebGL, 2D canvas, SVG and CSS3 represents WebGL at a high level using standard 3D graphics concepts http://www.tonyparisi.com 10/22/2 013
  20. 20. 3D animation requestAnimationFrame() http://www.paulirish.com/2011/requestanimationframe-forsmart-animating/ with JavaScript we can animate anything: materials, lights, textures…. Shaders Three.js has support for key frames, morphs and skins Tween.js – simple tweening library https://github.com/sole/tween.js/ var tween = new TWEEN.Tween( { x: 50, y: 0 } ) .to( { x: 400 }, 2000 ) .easing( TWEEN.Easing.Elastic.InOut ) .start(); function animate() { requestAnimationFrame( animate ); TWEEN.update(); } create and run tween call TWEEN.update() each frame to update values http://www.tonyparisi.com 10/22/2 013
  21. 21. creating the car (site) of the future http://www.tonyparisi.com 10/22/2 013
  22. 22. the concept design: TC Chang (http://www.tcchang.com/) http://www.tonyparisi.com 10/22/2 013
  23. 23. the model http://www.tonyparisi.com 10/22/2 013
  24. 24. the content pipeline - today still pretty crude – tools and converters under development sample work flows A. OBJ (single model) 1. 2. 3. create 3D model or import into Blender export to Three.js JSON format load into Three.js application; hand-layout, hand-light, hand-animate B. OBJ (single model) 1. 2. convert to Three.js JSON using Python command-line tool load into Three.js application; hand-layout, hand-light, hand-animate C. MD2/MD5 (Quake single model with animation) 1. 2. convert to Three.js JSON with online tool load into Three.js application; hand-layout, hand-light D. COLLADA (full scene) 1. 2. 3. 4. export to COLLADA from Maya, 3ds Max, Blender, Sketchup files contain scene layout, cameras, lights and animations – no need to do it by hand import into Three.js application and use but COLLADA files are big to download and slow to parse http://www.tonyparisi.com 10/22/2 013
  25. 25. the content pipeline - coming soon: glTF a “JPEG for 3D” bridges the gap between existing 3D formats/tools and today’s GL based APIs compact, efficient to load representation balanced, pragmatic feature set GL native data types require no additional processing also includes common 3D constructs (hierarchy, cameras, lights, common materials, animation ) reduces duplicated effort in content pipeline a common publishing format for content tools open specification; open process Khronos Group initiative within the COLLADA working group F. Robinet (lead), R. Arnaud, P. Cozzi, T. Parisi http://gltf.gl/ http://www.tonyparisi.com 10/22/2 013
  26. 26. exporting and previewing content tree view of 3D scene graph property sheets show detail scene/render stats cameras, lights and animations http://www.tonyparisi.com 10/22/2 013
  27. 27. Vizi: a framework for 3D applications component-based framework a la Unity3D Futurgo.prototype.go = function() { this.viewer = new Vizi.Viewer({ container : this.container, showGrid : true, allowPan: false, oneButton: true }); this.loadURL(Futurgo.URL); built-in Viewer object for this.viewer.run(); viewing models – handles } Futurgo.prototype.loadURL = function(url) { var that = this; var loader = new Vizi.Loader; loader.addEventListener("loaded", function(data) { that.onLoadComplete(data, loadStartTime); }); var loadStartTime = Date.now(); loader.loadScene(url); } Futurgo.prototype.onLoadComplete = function(data, loadStartTime) { var scene = data.scene; this.viewer.replaceScene(data); … Three.js setup, page events (e.g. resize), mouse events, run loop built-in Loader object loads models in Three.js JSON format, COLLADA, glTF add newly loaded model to scene http://www.tonyparisi.com 10/22/2 013
  28. 28. interaction and behaviors API methods use regular expression or object type to iterate over scene graph // Add entry fade behavior to the windows var that = this; scene.map(/windows_front|windows_rear/, function(o) { var fader = new Vizi.FadeBehavior({duration:2, opacity:.8}); o.addComponent(fader); setTimeout(function() { component model fader.start(); drop behaviors onto }, 2000); any object var picker = new Vizi.Picker; picker.addEventListener("mouseover", function(event) { that.onMouseOver("glass", event); }); picker.addEventListener("mouseout", function(event) { that.onMouseOut("glass", event); }); picker.addEventListener("touchstart", function(event) { that.onTouchStart("glass", event); }); picker.addEventListener("touchend", function(event) { pickers - dispatch mouse and that.onTouchEnd("glass", event); }); touch events to application o.addComponent(picker); 10/22/2 }); http://www.tonyparisi.com 013
  29. 29. animations Vizi Viewer object maintains list of named animations Futurgo.prototype.playAnimation = function(name, loop, reverse) { var animationNames = this.viewer.keyFrameAnimatorNames; loaded from var index = animationNames.indexOf(name); content file (e.g. if (index >= 0) COLLADA, glTF) { this.viewer.playAnimation(index, loop, reverse); } } Futurgo.prototype.playOpenAnimations = function() { this.playAnimation("animation_window_rear_open"); this.playAnimation("animation_window_front_open"); } play animations looped or unlooped, forward or reverse Futurgo.prototype.playCloseAnimations = function() { this.playAnimation("animation_window_rear_open", false, true); this.playAnimation("animation_window_front_open", false, true); } http://www.tonyparisi.com 10/22/2 013
  30. 30. integrating 2D and 3D 3D rollovers trigger 2D overlay display 2D page elements control 3D scene content http://www.tonyparisi.com 10/22/2 013
  31. 31. links game engines/IDEs libraries/frameworks PlayCanvas http://www.playcanvas.com/ Three.js Turbulenz https://turbulenz.com/ SceneJS Goo Enginehttp://www.gootechnologies.c om/ http://threejs.org/ http://scenejs.org/ BabylonJS http://www.babylonjs.com/ Artillery Engine https://artillery.com/ Voodoo.js http://www.voodoojs.com/ Verold http://verold.com/ tQuery Sketchfab https://sketchfab.com/ http://jeromeetienne.github.io/tquery/ Unreal… ? Vizi (VERY preliminary) http://www.extremetech.com/gaming/15190 0-unreal-engine-3-ported-to-javascript-andwebgl-works-in-any-modern-browser https://github.com/tparisi/Vizi http://www.tonyparisi.com 10/22/2 013
  32. 32. stay in touch… enjoy the show! http://html5devconf.com/ contact information tparisi@gmail.com skype: auradeluxe http://twitter.com/auradeluxe p://www.tonyparisi.com/ http://www.learningwebgl.com/ book source code https://github.com/tparisi/WebGLBook https://github.com/tparisi/Programming3DApplications htt get the books! WebGL: Up and Running http://www.amazon.com/dp/144932357X Programming 3D Applications with HTML and WebGL http://www.amazon.com/Programming-ApplicationsHTML5-WebGL-Visualization/dp/1449362966 http://www.tonyparisi.com 10/22/2 013

×