Glam - A Declarative Language for 3D Web Content

5,218 views
4,860 views

Published on

Slides from my talk at HTML5 Dev Conf 5/22/14 http://html5devconf.com/speakers/tony_parisi.html#session

Published in: Technology
1 Comment
7 Likes
Statistics
Notes
  • Hi all - also please visit the GLAM project page at http://www.glamjs.org/ or the Github repo at https://github.com/tparisi/glam
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
5,218
On SlideShare
0
From Embeds
0
Number of Embeds
296
Actions
Shares
0
Downloads
49
Comments
1
Likes
7
Embeds 0
No embeds

No notes for slide

Glam - A Declarative Language for 3D Web Content

  1. 1. A DECLARATIVE LANGUAGE FOR 3D WEB CONTENT TONY PARISI SEPTEMBER, 2014 GLAM:
  2. 2. CODE IS HARD. I LIKE MARKUP! THE WEB BROWSER IS NOW A 3D PLATFORM POWERFUL APIS PROVIDE EVERYTHING WE NEED: GRAPHICS, COMPOSITING, ANIMATION TIMING, THREADING, NETWORKING… INSANE 3D APPLICATIONS ARE NOW POSSIBLE API APPROACH PROVIDES MAXIMUM FLEXIBILITY – RENDER WHATEVER YOU WANT WITHOUT THE BROWSER NEEDING TO SUPPORT NEW OBJECTS BUT THE PENDULUM HAS SWUNG TOO FAR JAVASCRIPT MUCKING AND JSON HEFTING REQUIRED, EVEN FOR SIMPLE STUFF NO EASY WAY TO STYLE 3D CONTENT WE NEED A SIMPLE WAY TO CREATE 3D CONTENT IN PAGES THAT CAN BE STYLED AND PROGRAMMED LIKE ANY OTHER PAGE CONTENT 10/9/201 4 http://www.tonyparisi.com
  3. 3. DON’T BE SQUARE 10/9/201 4 http://www.tonyparisi.com LET’S MAKE A CUBE, BABY!
  4. 4. WebGwhatnow? 10/9/201 4 http://www.tonyparisi.com var vertexBuffer; vertexBuffer = gl.createBuffer(); gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer); var verts = [ // Front face -1.0, -1.0, 1.0, 1.0, -1.0, 1.0, 1.0, 1.0, 1.0, -1.0, 1.0, 1.0, // Back face -1.0, -1.0, -1.0, -1.0, 1.0, -1.0, 1.0, 1.0, -1.0, 1.0, -1.0, -1.0, … ]; gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(verts), gl.STATIC_DRAW); var vertexShaderSource = " attribute vec3 vertexPos;n" + " attribute vec2 texCoord;n" + " uniform mat4 modelViewMatrix;n" + " uniform mat4 projectionMatrix;n" + " varying vec2 vTexCoord;n" + " void main(void) {n" + " gl_Position = projectionMatrix * modelViewMatrix * n" + " vec4(vertexPos, 1.0);n" + " // Output the texture coordinate in vTexCoordn" + " vTexCoord = texCoord;n" + " }n"; var fragmentShaderSource = " precision mediump float;n" + " varying vec2 vTexCoord;n" + " uniform sampler2D uSampler;n" + " void main(void) {n" + " // Return the pixel color: always output whiten" + " gl_FragColor = texture2D(uSampler, vec2(vTexCoord.s…);n" + "}n"; function draw(gl, obj) { // clear the background (with black) gl.clearColor(0.0, 0.0, 0.0, 1.0); gl.enable(gl.DEPTH_TEST); gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT); // set the shader to use gl.useProgram(shaderProgram); // connect up the shader parameters: position, tex coord // projection/model matrices and texture // set up the buffers gl.bindBuffer(gl.ARRAY_BUFFER, obj.buffer); gl.vertexAttribPointer(shaderVertexPositionAttribute, obj.vertSize, gl.FLOAT, false, 0, 0) gl.bindBuffer(gl.ARRAY_BUFFER, obj.texCoordBuffer); gl.vertexAttribPointer(shaderTexCoordAttribute, obj.texCoordSize, gl.FLOAT, false, 0, 0 gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, obj.indices); gl.uniformMatrix4fv(shaderProjectionMatrixUniform, false, projectionMatrix); gl.uniformMatrix4fv(shaderModelViewMatrixUniform, false, modelViewMatrix); gl.activeTexture(gl.TEXTURE0); gl.bindTexture(gl.TEXTURE_2D, webGLTexture); gl.uniform1i(shaderSamplerUniform, 0); // draw the object gl.drawElements(obj.primtype, obj.nIndices, gl.UNSIGNED_SHORT, 0); } 1. CREATE BUFFERS 2. DEFINE VERTEX AND FRAGMENT SHADERS 3. DRAW THE CUBE 300 LINES OF JAVASCRIPT. UNIFORMS AND SHADY THINGS. NOT GROOVY.
  5. 5. Three.js A 3D LIBRARY WITH MOJO 10/9/201 4 http://www.tonyparisi.com renderer = new THREE.WebGLRenderer( { canvas: canvas, antialias: true } ); scene = new THREE.Scene(); camera = new THREE.PerspectiveCamera( 45, canvas.width / canvas.height, 1, 4000 ); scene.add(camera); var light = new THREE.DirectionalLight( 0xffffff, 1.5); scene.add( light ); var mapUrl ="../images/webgl-logo-256.jpg“; var map = THREE.ImageUtils.loadTexture(mapUrl ); var material = new THREE.MeshPhongMaterial({ map: map }); var geometry = new THREE.CubeGeometry(2, 2, 2); cube = new THREE.Mesh(geometry, material); scene.add( cube ); 1. CREATE RENDERER. CREATE SCENE. ADD TEXTURE MAP. DRAW CUBE. DONE. 40 LINES OF JAVASCRIPT. THAT’S COOL.
  6. 6. TURN AND FACE THE STRANGE IT TAKES 40 LINES OF JAVASCRIPT CODE TO MAKE A CUBE. ABOUT THAT, YES BUT THE CUBE SPINS, RIGHT? NOPE - THAT’S LOTS MORE LINES OF CODE NO MARKUP LANGUAGE? I HAVE TO WRITE CODE? YES BUT SURELY I CAN STYLE IT WITH CSS? ER… NO… THEY CAN PUT A MAN ON THE MOON (THEY WILL DO SOON!) BUT I CAN’T JUST SAY <cube> THAT’S MOST ODD.
  7. 7. INTRODUCING: GLAM GL AND MARKUP DECLARATIVE TAGS FOR DEFINING 3D VISUALS AND SCENE STRUCTURE SHAPES – 3D MESHES, LINES, SOLIDS, PATHS, EXTRUSIONS ANIMATIONS CAMERAS LIGHTS GROUPING AND SCENE HIERARCHY COMPLEX MODEL/SCENE IMPORT PARTICLES, POST-PROCESSING EFFECTS, VR RENDERING AND OTHER FUN EXTENDED CSS SYNTAX FOR STYLING AND ANIMATIONS DOM-LIKE API CREATING ELEMENTS CHANGING PROPERTIES HANDLING EVENTS DEFAULT BEHAVIORS BASED ON COMMON USAGE PATTERNS FULLY EXTENSIBLE USING JAVASCRIPT AND GLSL SHADERS RENDERED WITH WEBGL, BUILT WITH WEB COMPONENTS AND/OR POLYFILLS 10/9/201 4 http://www.tonyparisi.com
  8. 8. EIGHT LINE CUBE 10/9/201 4 http://www.tonyparisi.com <glam> <scene> <cube id="photocube”></cube> </scene> </glam> 1. THE HTML #photocube { image:url(../images/bowieorangeglam.png); } 2. THE CSS NOW THAT’S GLAM.
  9. 9. THE ASCENT OF GLAM 10/9/201 4 http://www.tonyparisi.com A BRIEF HISTORY OF GRAPHICS MARKUP VRML 1 VRML 2 X3D XAML GLAM 1994 1997 2004 2008 2014 STATIC MODELS, LO RES RENDERING +SIMPLE ANIMATION VRML + XML +COMPLEX ANIMATION, +HI RES (FOR WEB) RENDERING PLUGIN REQUIRED MARKUP, FULL PAGE INTEGRATION, LAME 3D IE ONLY GAME-QUALITY 3D, NO PLUGIN, FULL PAGE INTEGRATION AND NOW MARKUP! 1999 SVG
  10. 10. SHAPES AND MATERIALS 10/9/201 4 http://www.tonyparisi.com polygon meshes, geometry primitives, extruded text phong shading, specular highlights, wireframe rendering, transparency, textures <cube id="texturedcube" x="1.5" y='2' z='-5' depth='.5' height='1.5' ></cube> <sphere id="sphere1" radius='1.5'></sphere> <cone id="cone1" x='-1' y='-.5' z='2' rz='90deg'></cone> <cylinder id="cylinder1"></cylinder> <text id="text1" value="glam"></text> set visual properties via elements’ attributes – the DOM way!
  11. 11. GROUPS AND HIERARCHY 10/9/201 4 http://www.tonyparisi.com groups’ transform values apply to children <group y='-1'> <cube id="texturedcube" x="1.5" y='2' z='-5' depth='.5' height='1.5' ></cube> <group id="aGroup" x="1" y=".5" z="-2" > <sphere id="sphere1" radius='1.5'></sphere> <group id="nestedGroup" x='-3' y='1' z='-5'> <cone id="cone1" x='-1' y='-.5' z='2' rz='90deg'></cone> <cylinder id="cylinder1"></cylinder> </group> </group> <text id="text1" value="glam"></text> </group> groups can nest
  12. 12. INTERACTION 10/9/201 4 http://www.tonyparisi.com var photocube = document.getElementById('photocube'); photocube.addEventListener("mouseover", function(event) { stat.innerHTML = 'mouseover'; photocube.setAttribute('color-diffuse', 'red'); }); photocube.addEventListener("mouseout", function(event) { stat.innerHTML = 'mouseout'; photocube.setAttribute('color-diffuse', 'white'); }); photocube.addEventListener("mousedown", function(event) { stat.innerHTML = 'mousedown'; }); photocube.addEventListener("mouseup", function(event) { stat.innerHTML = 'mouseup'; }); photocube.addEventListener("mousemove", function(event) { stat.innerHTML = 'mousemove'; }); photocube.addEventListener("click", function(event) { stat.innerHTML = 'click'; }); mouse click on items in 3D generates DOM- style events; just add a listener to deal with it …JUST LIKE DOM USED TO MAKE!
  13. 13. ANIMATION 10/9/201 4 http://www.tonyparisi.com <scene> <cube id="photocube" animation="animRotateY"></cube> <animation id="animRotateY" duration="10s" iteration-count="infinite" timing-function="linear"> <keyframe time="0%" property="transform" value="rotateY(0deg);"></keyframe> <keyframe time="50%" property="transform" value="rotateY(180deg);"></keyframe> <keyframe time="100%" property="transform" value="rotateY(360deg);"></keyframe> </animation> </scene> OR target this node with named animation – similar to SVG animation @-webkit-keykeyframes kfRotateY { from { -webkit-transform: rotateY(0deg); } to { -webkit-transform: rotateY(360deg); } } .animRotateY { -webkit-animation-duration: 2s; -webkit-animation-name: kfRotateY; -webkit-animation-iteration-count: infinite; -webkit-animation-timing-function:linear; } animate 3D scene with CSS Animations!
  14. 14. STYLING 10/9/201 4 http://www.tonyparisi.com #photocube { image:url(../images/ash_uvgrid01.jpg); transform: translateX(-1.5) translateZ(-5) rotateY(30deg); } #cone1 { color:blue; radius:1; shader:phong; } #cylinder1 { color:green; radius:1; render-mode:wireframe; } #sphere1 { radius:1.5; shader:phong; diffuse-color:red; specular-color:#004400; opacity:.8; } transform properties (standard CSS3): translate, rotate, scale, skew, matrix texture map (custom property) visual CSS properties for materials size and position properties for shapes ALL WITH GOOD OLD CSS! TOO BAD THE BROWSER THROWS OUT UNKNOWN PROPERTIES… I HAD TO WRITE A CUSTOM PARSER! (BASED ON JQUERY PLUGIN)
  15. 15. IMPORTING MODELS 10/9/201 4 http://www.tonyparisi.com <import src="../models/futurgo_city/futurgo_city.dae”></import> EMBED POPULAR/STANDARD 3D FORMATS WE CAN’T LIVE ON CUBES ALONE… MODEL FORMATS – OBJ, STL SIMPLE MODELS, NO SCENE HIERARCHY, CAMERAS, LIGHTS OR ANIMATION ANIMATION FORMATS – BVH, MD2, MD5 SCENE FORMATS – COLLADA, glTF SUPPORT FULL SCENE STRUCTURE, CAMERAS, LIGHTS, ANIMATIONS
  16. 16. SHADERS AND EXTENSIBILITY WE CAN’T THINK OF EVERY POSSIBLE COOL SHAPE, MATERIAL AND BEHAVIOR GLSL SHADER LANGUAGE LETS US CONTROL EVERY VERTEX AND PIXEL 10/9/201 4 http://www.tonyparisi.com .bubble { radius:.5; color:lightgray; vertex-shader:url(../shaders/fresnel.vs); fragment-shader:url(../shaders/fresnel.fs); shader-uniforms:mRefractionRatio f 1.02 mFresnelBias f 0.1 mFresnelPower f 2.0 mFresnelScale f 1.0 tCube t null; } declare vertex and fragment shaders in CSS declare shader language inputs so engine knows how to pass in values
  17. 17. DOM API 10/9/201 4 http://www.tonyparisi.com function createBubble(scene, x, y, z, animClass) { var groupelt = document.createElement('group'); groupelt.setAttribute('x', x); groupelt.setAttribute('z', z); groupelt.setAttribute('y', y); var sphereelt = document.createElement('sphere'); sphereelt.setAttribute('class', "bubble skybox"); sphereelt.addEventListener("click", function(event) { pop.pause(); pop.currentTime = 0; pop.play(); scene.removeChild(groupelt); }); sphereelt.setAttribute('animation', animClass); groupelt.appendChild(sphereelt); return groupelt; } mouse click on items in 3D generates DOM-style events; just add a listener to deal with it use standard DOM methods to add/remove children get/set values using standard DOM method
  18. 18. POST-PROCESSING AND EFFECTS PARTICLE SYSTEMS… IN JUST A FEW TAGS PIXELATE, BLUR, SCREEN, DEPTH OF FIELD… IN JUST A FEW TAGS 10/9/201 4 http://www.tonyparisi.com <particles maxAge='2' id="fire"> <emitter particlesPerSecond='50' size='10' sizeEnd='5' opacityStart='0.2’ opacityMiddle='0.4' opacityEnd='0.1' velocity='0 2.5 0' acceleration='0 3 0' positionSpread='2 3.5 2' accelerationSpread='1.5 .5 1.5' colorStart="yellow blending='additive' > </emitter> </particles> particle system age and emitter params are defined in the markup post-processing passes affect the whole scene #fire { image:url(../../images/smokeparticle.png); } particle sprite is defined in the CSS <scene> <cube id="photocube" class=""></cube> <effect type="RGBShift" amount=".005"> </effect> <effect type="DotScreenRGB" scale='2'> </effect> </scene>
  19. 19. VR + ML 10/9/201 4 http://www.tonyparisi.com <glam> <renderer type="rift"></renderer> <scene> <controller type="rift"></controller> <background id="sb1" class="skybox"></background> <group y ='1' z='-3'> <sphere class="bubble skybox”></sphere> <sphere x='-1' z='-2' class="bubble skybox”></sphere> </group> </scene> </glam> set up VR renderer set up VR controller to track head motion
  20. 20. IMPLEMENTATION (I) 10/9/201 4 http://www.tonyparisi.com THE RENDERING/ANIMATION STACK VIZI – A COMPONENT-BASED FRAMEWORK FOR BUILDING INTERACTIVE 3D APPLICATIONS https://github.com/tparisi/vizi THREE.JS – A JAVASCRIPT 3D RENDERING LIBRARY/ENGINE https://github.com/mrdoob/three.js/ TWEEN.JS – A SIMPLE BUT POWERFUL TWEENING UTILITY https://github.com/sole/tween.js
  21. 21. IMPLEMENTATION (II) 10/9/201 4 http://www.tonyparisi.com BROWSER STUFF BROWSER PARSES UNKNOWN DOM ELEMENTS; DOESN’T KNOW WHAT TO DO WITH THE DOM TREE BUT SAVES IT ANYWAY WE TRAVERSE THAT TREE AND BUILD 3D RENDERING/ANIMATION STRUCTURES WE USE MUTATION OBSERVERS TO MONITOR CHANGES TO THE DOM AND UPDATE THE LIVE SCENE GRAPH BROWSER IGNORES UNKNOWN CSS GLAM WAS BUILT WITH A JQUERY CSS PARSER PLUGIN
  22. 22. IMPLEMENTATION (III) 10/9/2014http://www.tonyparisi.com WHAT ABOUT WEB COMPONENTS? LOVE ‘EM. THEY LOOK LIKE THE FUTURE. BUT I STARTED THIS PROJECT BEFORE THEY WERE GETTING TRACTION. NOW I NEED TO GO BACK AND REWORK THIS TO USE THEM. BUT… WEB COMPONENTS AREN’T SUPPORTED IN RETAIL BROWSERS YET – YOU NEED NIGHTLIES WE CAN POLYFILL WITH POLYMER… BUT IT’S HARDCORE AND I'M A SOFTIE http://www.polymer-project.org/ BUT… BUT… BUT… WEB COMPONENTS-Y PEOPLE SEEM TO BE AVERSE TO EXTENDING CSS*. MAYBE IT’S JUST ME BUT I THINK 3D NEEDS NEW ATTRIBUTES (Z, DEPTH, RADIUS, NORMAL-IMAGE, SPECULAR-COLOR ETC.) POLYMER TAKES A HANDS-OFF ATTITUDE – JUST DO EVERYTHING IN ELEMENT ATTRIBUTES, THE “NEW WAY?” * WHAT’S THE BLOODY POINT OF BUILDING AN EXTENSIBLE MARKUP SYSTEM THAT IGNORES THE BLOODY STYLING? BLOODY HELL…
  23. 23. PROJECT STATUS WE ARE AT ABOUT V0.7… NEEDS REVIEW, REFACTORING AND DESIGN INPUT WE NEED MORE DEMOS AND EXAMPLES, DOCUMENTATION AND WE NEED CONTRIBUTORS! PORT TO WEB COMPONENTS/POLYMER I STARTED THIS PROJECT BEFORE WEB COMPONENTS WAS TAKING OFF; IT’S TIME TO REFACTOR IT SO AS TO NOT REINVENT WHEELS. DISCUSS WITH BROWSER MAKERS– WE COULD USE HELP WITH CSS EXTENSIBILITY. WE NEED THOSE CUSTOM PROPERTIES. I THINK THIS IS CRITICAL; BUT IS IT JUST ME? CAN YOU DIG IT? DECLARE YOURSELF: GET INVOLVED. HELP ME MAKE THIS REAL. 10/9/201 4 http://www.tonyparisi.com
  24. 24. http://www.tonyparisi.com 10/9/201 4 CAN YOU DIG IT? CONTACT tony@vizi.gl skype: auradeluxe http://twitter.com/auradeluxe http://www.tonypa risi.com/ http://www.learningwebgl.com/ GET THE BOOKS! WebGL: Up and Running http://www.amazon.com/dp/144932357X Programming 3D Applications with HTML and WebGL http://www.amazon.com/Programming-Applications-HTML5-WebGL- Visualization/dp/1449362966 GET VIZI https://github.com/tparisi/Vizi MEETUPS http://www.meetup.com/WebGL-Developers-Meetup/ http://www.meetup.com/Web-VR/ BOOK CODE https://github.com/tparisi/WebGLBook https://github.com/tparisi/Programming3DApplications GET GLAM http://www.glamjs.org/ https://github.com/tparisi/glam/ WORK http://www.vizi.gl/ CREDS Co-creator, VRML and X3D

×