A brief introduction to three.js & Leap Motion in the browser.

1. three.js
& leap motion
An introduction & overview of 3D graphics in the browser

2. • 3D primer
• What is three.js
• First steps with three.js
• What is a Leap Motion Controller
• Leap demo
• Washington Post special project review
Overview

3. This is not a tutorial.3D graphics in and of itself is a very complicated subject. This presentation is not a tutorial
but does attempt to cover (very briefly) the basics and help you understand where to get
started with three.js & how things fit together. With that said- the documentation around
three.js is very good and it’s fairly easy to find tutorials online.

4. A simple scene
consisting of a
light, cube, plane
and camera.
An Example
3D Scene
3D Primer
• The lights, cameras & objects are collectively called
a scene. In three.js it is a hierarchy & the scene is
the root of the tree.
• Objects or geometry such as cubes and planes (also
called primitives) are created by points in space
called vertices.
• A group of vertices and their edges create polygons.
• A group of polygons (objects) along with a material
create a mesh.
• The camera captures the picture and in the example
to the left we are looking through the camera.
• Lights allow us to cast shadows and render
highlights.

5. First Steps
with
three.js
• Follow the three.js “Getting Started” section in the docs.
https://threejs.org/docs/index.html#Manual/Introduction/Creating_a_scene
• Load a custom model instead of using geometry.
• Add a light & cast shadows.

6. three.js
example

7. .
├── index.html
├── js
│ ├── ext
│ │ ├── three.js
│ │ └── three.modules.js
│ └── main.js
└── run.py
directory structure

8. 1 console.info("main.js loaded");
2
3 var scene = new THREE.Scene();
4 var camera = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, 0.1, 1000);
5 console.info("scene & camera created");
6
7 var renderer = new THREE.WebGLRenderer();
8 renderer.setSize(window.innerWidth, window.innerHeight);
9 console.info("renderer created");
10
11 document.body.appendChild(renderer.domElement);
12 console.info("DOM modified");
13
14 var geometry = new THREE.BoxGeometry(1, 1, 1);
15 var material = new THREE.MeshBasicMaterial({color: 0x00ff00});
16 var cube = new THREE.Mesh(geometry, material);
17 scene.add(cube);
18 console.info("Cube created");
19
20 camera.position.z = 5;
21 console.info("Camera repositioned");
22
23 function render() {
24 // Animate the cube
25 cube.rotation.x += 0.01;
26 cube.rotation.y += 0.01;
27 requestAnimationFrame(render);
28 renderer.render(scene, camera);
29 }
30 console.info("Starting render")
31 render();

9. demo

10. Free “digger” model from
TurboSquid.
loading
custom
models

11. Model geometry must be
triangulated.
Note how there are now
diagonal lines and all of
the surfaces have
triangles.

12. Sidebar:
Use a 3D graphics program to
adjust your model’s geometry.
Any 3D program can adjust geometry. Blender
is free and can easily do this for us.

13. Loading
JSON
• Use the free utility from the three.js project to convert from
OBJ to JSON.
https://github.com/alteredq/three.js/blob/master/utils/exporters/convert_obj_three.py
• Use the built-in JSONLoader to load the object.
• Watch file sizes! If you are using a “real” web server
enable gzip compression!

14. 1 var greenMat = new THREE.MeshBasicMaterial({color: 0x00ff00});
2
3 // Create a JSON loader
4 var loader = new THREE.JSONLoader();
5
6 // Load triangulated model as JSON
7 loader.load(
8 'models/digger_scaled_tri.json',
9 // onLoad callback
10 function (geometry) {
11 // Add geometry to a new mesh combining a material
12 var object = new THREE.Mesh(geometry, greenMat);
13 // Add our object to the scene
14 scene.add(object);
15 }
16 );

15. demo

16. lighting &
shadows

17. three.js has many types of
lights but we’ll focus on the
directional light.
• Directional lights shine from a specific direction
not a specific location.
• The documentation describes this as a light that
acts like the sun.
• Shadows are only calculated for objects that
exist within the light’s shadow camera’s field of
view.
• The shadow camera for a directional light is
orthographic (think 2D, e.g. top down).
• In the picture to the left the orange lines denote
the shadow camera’s field of view.

18. three.js uses shadow
mapping with the WebGL
renderer.
• Shadow mapping was introduced by Lance
Williams in 1978.
• Computation takes place on the GPU with
WebGL.
• Think of a shadow map like a texture map. They
are applied (mapped) to the surface of the
geometry during rendering.
• The size of the map directly affects the shadow
detail because the size of the map is the size of
the buffer and a larger map/buffer can hold more
information about the shadow.
• Shadows aren’t necessarily hard but they
require some tweaking and experimenting to get
right.

19. demo

20. The Leap Motion Controller lets you use your computer in a whole new way.
Reach out and swipe, grab, pinch, or punch your way through the digital world.
• Uses two wide angle IR cameras to detect hands in 3D space above the device.
• Tracks hands, fingers and wrists / arms and translates motion from real world
space to 3D environment coordinates.
• For developers it is plug & play and very accessible.
• It is a bit finicky and detecting / supporting custom gestures can be laborious.
Leap Motion Controller

21. demo

22. In 2015 I was contracted to help build a 3D interactive
experience exploring the restoration of the U.S. Capitol
Dome.
The interactive experience took place at the White House
Correspondents Dinner. The scope was narrowed down to
build a kiosk where visitors could approach one of two large
displays and interact with a 3D model of the dome using a
Leap Motion controller.
Washington Post
Special Project

23. 3D
Model
three.js enabled the rapid
development & delivery of
the experience within a
browser.
Many issues could be ignored because we were operating as a kiosk
in a known environment:
• Device hardware specs were known (fast enough to ignore
optimizations)
• No external bandwidth requirements (all files are hosted & served
locally)
• Human explanations & help to explain how to interact with the
demo.
• No need to open up “heavier” tooling such as Unity.
Leap
Motion
Web
Browser
three.js

24. early prototype

25. Things didn’t go as planned

26. 3 Weeks to Deliver
Interaction
Spent the majority of the time
figuring out gestures and hand
controls. Tried “gimbal” controls.
Settled on “joystick” controls.
“Hot Spots” & Polishing
Used ray casting for “fast enough”
collision detection between hands
and “hot spots”. Added dynamic
“demo” mode to pique interest.
Rendering & Lighting
Lost a good chunk of time to loading
a model of the entire capital.
Reverted to just the dome. Had
trouble with shadows & model size.

27. demo

28. THANK YOU!
@leetrout