WT-4064, Build Rich Applications with HTML5 and WebGL, by Tony Parisi

BUILD RICH APPLICATIONS IN HTML5
AND WEBGL
TONY PARISI
NOVEMBER 12, 2013

ABOUT ME
 serial entrepreneur

 founder, Vizi (stealth startup)
 consulting architect and CTO
 co-creator, VRML and X3D web standards
 co-designer, glTF
 author, speaker instructor
 contact information

book source code
https://github.com/tparisi/WebGLBook
https://github.com/tparisi/Programming3DApplications

tparisi@gmail.com
skype: auradeluxe
http://twitter.com/auradeluxe
http://www.tonyparisi.com/
http://www.learningwebgl.com/

2 | BUILD RICH APPLICATION S IN HTML5 AND WEBGL | NOVEMBER 21, 2013


RIP: 1995-2013

Image:


Eric Dye


LONG LIVE…



HTML5: THE BROWSER AS PLATFORM
 graphics APIs– Canvas, WebGL, SVG
 CSS3 – animations, transitions, transforms and filter effects
 hardware-accelerated compositing
‒ all elements blend seamlessly on page

 JavaScript performance and language enhancements
 and a whole lot more…
‒ first-class streaming media types - <audio>, <video>
‒ device input
‒ location and mobile-inspired features
‒ database, local storage, file system
‒ networking – sockets and real-time connections (WebRTC)
‒ multi-threading (Workers)



WebGL: REAL-TIME 3D RENDERING

 the 3D API standard
‒ OpenGL ES™ in a browser
‒ JavaScript API bindings
‒ supported in all modern browsers
‒ shipped since early 2011

…and it’s
awesome.

supported in
•
•
•
•

desktop Safari, Firefox, Chrome, Opera, IE
Android – mobile Chrome, mobile Firefox
FireOS (Kindle Fire HDX)
Blackberry, Tizen, Firefox OS

•

Surface (Windows 8.1)

• iOS mobile Safari – iAds only
• 500M+ seats -> 1B

100,000 Stars Google Experiment


WebGL APPLICATIONS
advertising and media

data visualization

products and e-commerce

page graphics

JUST HOW RICH?

Epic Games’ Unreal Citadel Running in Firefox
http://www.unrealengine.com/html5/

60FPS

ported in 5 days

Unreal native C++ engine -> JavaScript


Emscripten + asm.js

HOW WebGL WORKS

 it’s a JavaScript drawing API
‒ draw to a canvas element using a special context (“webgl”)
‒ low-level drawing – buffers, primitives, textures and shaders
‒ accelerated by graphics hardware (GPU)
‒ can draw 2D as well as 3D graphics

 integrates seamlessly with other page content
 there is no file format; no markup language; no DOM.
 libraries and frameworks are key to fast ramp up and productive development



A SIMPLE WebGL PROGRAM

1.

create a <canvas> element

2.

obtain a drawing context

3.

initialize the viewport

4.

create one or more buffers

5.

create one or more matrices

6.

create one or more shaders

7.

initialize the shaders

8.

draw one or more primitives



CREATE THE CANVAS, CONTEXT AND VIEWPORT
function initWebGL(canvas) {
var gl = null;
var msg = "Your browser does not support WebGL, " +
"or it is not enabled by default.";
try
{
gl = canvas.getContext(“webgl");
}
catch (e)
{
msg = "Error creating WebGL Context!: " + e.toString();
}

detect WebGL

if (!gl)
{
alert(msg);
throw new Error(msg);
}
return gl;

set WebGL drawing region

}
function initViewport(gl, canvas)
{
gl.viewport(0, 0, canvas.width, canvas.height);
}



BUFFERS AND TYPED ARRAYS
var vertexBuffer;
vertexBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer);
var verts = [
// Front face
-1.0, -1.0, 1.0,
1.0, -1.0, 1.0,
1.0, 1.0, 1.0,
-1.0, 1.0, 1.0,
// Back face
-1.0, -1.0, -1.0,
-1.0, 1.0, -1.0,
1.0, 1.0, -1.0,
1.0, -1.0, -1.0,
…

WebGL drawing functions use buffers
of data

Typed Arrays:
new low-level data type stores
arrays of floats and ints compactly

];
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(verts), gl.STATIC_DRAW);


SHADERS
var vertexShaderSource =
"
"
"
"
"
"
"
"
"
"
"
"

attribute vec3 vertexPos;n" +
attribute vec2 texCoord;n" +
uniform mat4 modelViewMatrix;n" +
uniform mat4 projectionMatrix;n" +
varying vec2 vTexCoord;n" +
void main(void) {n" +
// Return the transformed and projected vertex valuen" +
gl_Position = projectionMatrix * modelViewMatrix * n" +
vec4(vertexPos, 1.0);n" +
// Output the texture coordinate in vTexCoordn" +
vTexCoord = texCoord;n" +
}n";

var fragmentShaderSource =
" precision mediump float;n" +
" varying vec2 vTexCoord;n" +
" uniform sampler2D uSampler;n" +
" void main(void) {n" +
" // Return the pixel color: always output whiten" +
" gl_FragColor = texture2D(uSampler, vec2(vTexCoord.s, vTexCoord.t));n" +
"}n";

the vertex shader transforms modelspace positions into screen space

the fragment shader outputs a
color value for each pixel


DRAWING
function draw(gl, obj) {
// clear the background (with black)
gl.clearColor(0.0, 0.0, 0.0, 1.0);
gl.enable(gl.DEPTH_TEST);
gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);

clear the canvas

// set the shader to use
gl.useProgram(shaderProgram);

set the shader

// connect up the shader parameters: vertex position, texture coordinate,
// projection/model matrices and texture
// set up the buffers
gl.bindBuffer(gl.ARRAY_BUFFER, obj.buffer);
gl.vertexAttribPointer(shaderVertexPositionAttribute, obj.vertSize, gl.FLOAT, false, 0, 0);
gl.bindBuffer(gl.ARRAY_BUFFER, obj.texCoordBuffer);
gl.vertexAttribPointer(shaderTexCoordAttribute, obj.texCoordSize, gl.FLOAT, false, 0, 0);
gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, obj.indices);
gl.uniformMatrix4fv(shaderProjectionMatrixUniform, false, projectionMatrix);
gl.uniformMatrix4fv(shaderModelViewMatrixUniform, false, modelViewMatrix);
gl.activeTexture(gl.TEXTURE0);
gl.bindTexture(gl.TEXTURE_2D, webGLTexture);
gl.uniform1i(shaderSamplerUniform, 0);
// draw the object
gl.drawElements(obj.primtype, obj.nIndices, gl.UNSIGNED_SHORT, 0);
}

set up buffers for vertices
and texture coordinates

pass transform and
projection matrices
to the shader
set the texture and pass to the shader
draw the object

A WebGL CLIENT-SIDE STACK

 rendering – Three.js library
 animation – Three.js, Tween.js libraries
 application functionality – game engine/framework
‒ setup/teardown
‒ interaction – picking, rollovers, rotating models
‒ behaviors
‒ run loop and event dispatching

 content creation pipeline
‒ 3D tools e.g. Autodesk Maya
‒ COLLADA, glTF formats; conversion tools



Three.js: A JAVASCRIPT 3D ENGINE
https://github.com/mrdoob/three.js/

 the most popular WebGL library
renderer = new THREE.WebGLRenderer( { canvas: canvas, antialias: true } );
scene = new THREE.Scene();
camera = new THREE.PerspectiveCamera( 45, canvas.width /
canvas.height, 1, 4000 );
scene.add(camera);

can render to WebGL,
2D canvas, SVG and CSS3

var light = new THREE.DirectionalLight( 0xffffff, 1.5);
scene.add( light );
var mapUrl ="../images/webgl-logo-256.jpg“;
var map = THREE.ImageUtils.loadTexture(mapUrl );
var material = new THREE.MeshPhongMaterial({ map: map });

represents WebGL at a
high level using standard
3D graphics concepts

var geometry = new THREE.CubeGeometry(2, 2, 2);
cube = new THREE.Mesh(geometry, material);
scene.add( cube );


3D ANIMATION
 requestAnimationFrame()
http://www.paulirish.com/2011/requestanimationframe-for-smartanimating/

 with JavaScript we can animate anything: materials, lights,
textures…. Shaders
 Three.js has support for key frames, morphs and skins
 Tween.js – simple tweening library
https://github.com/sole/tween.js/
var tween = new TWEEN.Tween( { x: 50, y: 0 } )
.to( { x: 400 }, 2000 )
.easing( TWEEN.Easing.Elastic.InOut )
.start();
function animate() {
requestAnimationFrame( animate );
TWEEN.update();
}

create and
run tween
call TWEEN.update()
each frame to update values

WebGL PIPELINE TOOLS

 WebGL has no file format; no markup language; no DOM.
‒ apps/libraries must define their own formats

 a lot of people are still creating content by writing code
 existing 3D formats proprietary, incomplete or ill-suited for web/mobile
delivery
 help is on the way: glTF
‒ a “JPEG for 3D”

‒ bridges the gap between existing 3D formats/tools and today’s GL
based APIs
‒ compact, efficient to load representation
‒ JSON scene description and high-level objects
‒ binary vector and animation data

 a common publishing format for content tools
‒ open specification; open process

glTF Three.js Loader Prototype

‒ Khronos Group initiative within the COLLADA working group
‒ F. Robinet (lead), R. Arnaud, P. Cozzi, T. Parisi
‒ https://github.com/KhronosGroup/glTF
‒ http://gltf.gl/


WebGL ENGINES AND FRAMEWORKS
 Provide application-level functionality
‒ setup/teardown
‒ interaction – picking, rollovers, rotating
models
‒ behaviors
‒ run loop and event dispatching
‒ FSM objects (states)
‒ physics, dynamics, collision, …
‒ view models, camera controllers, …

game engines/IDEs

libraries/frameworks

PlayCanvas http://www.playcanvas.com/

Three.js
http://threejs.org/

Turbulenz https://turbulenz.com/
SceneJS
Goo Enginehttp://www.gootechnologies.com/

http://scenejs.org/

Artillery Engine https://artillery.com/

BabylonJS

Verold http://verold.com/
Sketchfab https://sketchfab.com/
Unreal… ?
http://www.extremetech.com/gaming/151900unreal-engine-3-ported-to-javascript-and-webglworks-in-any-modern-browser

http://www.babylonjs.com/

Voodoo.js
http://www.voodoojs.com/

tQuery
http://jeromeetienne.github.io/tquery/

Vizi
https://github.com/tparisi/Vizi



CSS3: ADVANCED PAGE EFFECTS

 CSS transitions
 CSS animations
 CSS 3D transforms

 CSS filter effects
 CSS custom filters



CSS3 TRANSITIONS
source: http://www.kirupa.com/html5/all_about_css_transitions.htm

 allow gradual changes to CSS properties over time
new CSS3 properties
currently require vendorspecific prefixes

#box img {
-webkit-transition: transform .5s ease-in;
-webkit-transform: translate3d(0, -350px, 0);
}
#box img:hover {
-webkit-transition: transform .5s ease-in;
-webkit-transform: translate3d(0, 0px, 0);
cursor: pointer;
}

-webkit-transition: all .5s ease-in -.1s;


shortcut for

specify property to
transition, duration of
transition, easing
function

transition-property: all;
transition-duration: .5s;
transition-timing-function: ease-in;
transition-delay: .1s;

CSS3 ANIMATIONS
source: http://www.kirupa.com/html5/all_about_css_animations.htm

 allow fine control over animation behavior with key frames
#bigcloud {
animation: bobble 2s infinite;
}
@keyframes bobble {
0% {
transform: translate3d(50px, 40px, 0px);
animation-timing-function: ease-in;
}
50% {
animation-timing-function: ease-out;
}
100% {
}
}

each key frame contains a transform

control timing with
ease in/out functions
keys are used to interpolate
transform values

CSS3 3D TRANSFORMS

 translate, rotate, scale page elements with perspective
browser will render element in 3D perspective

.bk-list li {

perspective: 1800px;
}
apply to child
elements

.bk-list li .bk-front {
transform-style: preserve-3d;
add origin to
transform-origin: 0% 50%;
translation
transform: translate3d(0,0,20px);
}

apply 35 degree rotation about Y axis
.bk-list li .bk-book.bk-bookdefault:hover {
transform: rotate3d(0,1,0,35deg);
}

Using 3D
transform forces
hardware
acceleration

http://tympanus.net/codrops/2013/01/08/3dbook-showcase/


CSS3 FILTER EFFECTS
source: http://html5-demos.appspot.com/static/css/filters/index.html

 apply post-processing effects to image elements
img {
-webkit-filter: sepia(100%);
}
img {
-webkit-filter: grayscale(100%);
}
img {
-webkit-filter: blur(2px);
}
img {
-webkit-filter: brightness(33%);
}
img {
-webkit-filter: contrast(67%);
}


CSS3 CUSTOM FILTERS
source: http://alteredqualia.com/css-shaders/crumple.html
.shader {
-webkit-filter: custom(url(shaders/crumple.vs) mix(url(shaders/crumple.fs) multiply source-atop), 50 50, amount 0, strength 0.2,
lightIntensity 1.05, transform rotateX(0deg) translateZ(0px) );
-webkit-transition: -webkit-filter linear 1.5s;
box-shadow: 0 0 2em #111;
}
.shader:hover {
-webkit-filter: custom(url(shaders/crumple.vs) mix(url(shaders/crumple.fs) multiply source-atop), 50 50, amount 1, strength 0.2,
lightIntensity 1.05, transform rotateX(0deg) translateZ(0px) );
}

Crumple shader by
Branislav Ulicny (AlteredQualia)
http://alteredqualia.com/cssshaders/crumple.html



CSS3 VERTEX SHADER
void main() {
// Compute displacement
float time = 10.0 * quadraticInOut( amount );
vec2 surfaceCoordinate = 0.025 * a_texCoord;
float n = surface( surfaceCoordinate, time );
float curve = strength * n;

browser predefines a_texCoord
and a_position, passes into
shader

vec4 pos = a_position;
pos.z = amount * ( 0.5 * strength - curve );
// Compute normal
const float e = 0.001;
float nx = surface( surfaceCoordinate + vec2( e, 0.0 ), time );
float ny = surface( surfaceCoordinate + vec2( 0.0, e ), time );
vec3 normal = normalize( vec3( n - nx, n - ny, 1.0 - strength * 1.25 ) );
// Compute lighting (directional light)
vec3 lightPosition = normalize( vec3( 0.0, 0.5, 1.0 ) );
float lightWeight = lightIntensity * max( dot( normal, lightPosition ), 0.0 );
// Set vertex position
gl_Position = u_projectionMatrix * perspective( 0.9 ) * transform * pos;

shader outputs new
vertex position,
passes lighting
information to
fragment shader

v_uv = a_texCoord;
v_height = n;
v_light = lightWeight;


CSS3 FRAGMENT SHADER

// Uniform values from CSS
uniform float amount;

// Varyings passed in from vertex shader
varying vec2 v_uv;
varying float v_height;
varying float v_light;
void main() {
const float a = 1.0;
float r, g, b;
// Light variant
float n = v_light;
float v = mix( 1.0, n * n, amount );
r = g = b = sqrt( v );
// Set color matrix

Fragment shader
outputs blend value
based on light
calculated in vertex
shader and usersupplied amount

css_ColorMatrix = mat4( r, 0.0, 0.0, 0.0,
0.0, g, 0.0, 0.0,
0.0, 0.0, b, 0.0,
0.0, 0.0, 0.0, a );
}



CSS3 SUPPORT
source: http://css3.bradshawenterprises.com/

CSS3 Transitions
supported since
• Safari 3.2: 13/11/2008
• Firefox 4.0: Late 2010
• Chrome 1.0: 02/09/2008
• Opera 10.5: 02/03/2010
• Internet Explorer 10: 09/2011

CSS 3D Transforms
supported since
• Safari 4.0: 11/06/2008
• Chrome: 28/08/2010
• IE 10: 09/2011
• Firefox: 27/10/2011

CSS3 Filters
CSS3 Animations
supported since
• Safari 4.0: 11/06/2008
• Chrome 1.0: 02/09/2008
• Firefox 5: 20/04/2011
• IE 10: 09/2011
• Opera: 03/2012

supported since
• Safari 6.0
• Chrome: 18.0

CSS3 Custom Filters
supported only in Chrome

FUN WITH THREE.JS AND CSS3

http://mrdoob.github.io/three.js/examples/css3d_periodictable.html


LANGUAGE INNOVATIONS
tackling performance and memory challenges

 asm.js
optimizable, low-level subset of JavaScript; strictly typed; improves performance (2x native vs. 3-10x) runs
in FF and Chrome nightlies
http://asmjs.org/

 Emscripten
LLVM-to-JavaScript compiler translates C++ native code to JavaScript; can output to asm.js
https://github.com/kripken/emscripten/wiki

 new Web languages - Dart, TypeScript
optionally strongly typed; class-based
https://www.dartlang.org/
http://www.typescriptlang.org/



RESOURCES





WebGL specification http://www.khronos.org/registry/webgl/specs/latest/
Learning WebGL http://www.learningwebgl.com/
Three.js https://github.com/mrdoob/three.js/
requestAnimationFrame
http://www.paulirish.com/2011/requestanimationframe-for-smart-animating/

 CSS specifications
http://www.w3.org/TR/css3-animations/
http://www.w3.org/TR/css3-transitions/
http://www.w3.org/TR/css3-transforms/
http://www.w3.org/TR/filter-effects/

 CSS animation demos http://www.creativebloq.com/css3/animation-with-css3-712437
 CSS 3D transforms explained http://desandro.github.io/3dtransforms/
 Adobe filter lab - playing with filter
effectshttp://html.adobe.com/webplatform/graphics/customfilters/cssfilterlab/
 Alan Greenblatt’s blog – CSS filter effects http://blattchat.com/


STAY IN TOUCH…

contact information
tparisi@gmail.com
skype: auradeluxe
http://twitter.com/auradeluxe
arisi.com/
http://www.learningwebgl.com/

http://www.tonyp

book source code
https://github.com/tparisi/WebGLBook
https://github.com/tparisi/Programming3DApplications

get the books!
WebGL: Up and Running
http://www.amazon.com/dp/144932357X
Programming 3D Applications with HTML and WebGL
http://www.amazon.com/Programming-Applications-HTML5-WebGLVisualization/dp/1449362966


WT-4064, Build Rich Applications with HTML5 and WebGL, by Tony Parisi

More Related Content

What's hot

Similar to WT-4064, Build Rich Applications with HTML5 and WebGL, by Tony Parisi

More from AMD Developer Central

Recently uploaded

WT-4064, Build Rich Applications with HTML5 and WebGL, by Tony Parisi