Getting Started in VR with JS

Getting Started in VR with JS
The Dream of the 90s is Alive!
#empirejs2015

Why would this time be any
different?

JavaScript FTW
WebVR allows us to work in JS “native” browser
environment.

What’s a headset?
WOAH!
your app renders
1. delivers position data to
2. surface to display
stereoscopic image
.js

A device that gives positional
data & a surface to draw on?
Sounds awful like a smart phone!

I’ll be giving these out
Come see me in the Q&A lounge after this and
throughout the conference.

Demos available at c5vr.com
And also in the Q&A Lounge after.

Download ALL THE THINGS!
Drivers and SDK from developer.oculus.com
Firefox WebVR Browser from mozvr.com/downloads
Chromium WebVR Browser from bit.ly/1DPjgDQ

Not a WebGL replacement
WebGL is a framework to build your own 3D graphics
rendering pipeline.

Math is HARD
Computing per pixel transform and coloring takes a lot
of work.

Use three.js
Provides a higher level abstraction that is easier to work
with.

no_vr.html
<!DOCTYPE html>
<html lang="en">
<head>
<title>No VR</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, user-scalable=no, minimum-scale=1.0, maximum-scale=1.0">
<meta name="mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-capable" content="yes" />
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />
<style>
body {
background-color: #000;
color: #fff;
margin: 0px;
padding: 0;
overflow: hidden;
}
</style>
</head>
<body>
<script src="js/three.js"></script>
<script src="js/no_vr.js"></script>
</body>
</html>

no_vr.html
</style>
</head>
<body>
<script src="js/no_vr.js"></script>
</body>
</html>

no_vr.js - Create a Renderer
var renderer = new THREE.WebGLRenderer({antialias: true});
renderer.setPixelRatio(window.devicePixelRatio);
renderer.setSize(window.innerWidth, window.innerHeight);
document.body.appendChild(renderer.domElement);
var scene = new THREE.Scene();
var camera = new THREE.PerspectiveCamera(
75, window.innerWidth / window.innerHeight, 0.3, 10000
);
var light = new THREE.PointLight(0xffffff, 1.0, 0);
light.position.set(0,0,0);
scene.add(light);
var bitGeometry = new THREE.DodecahedronGeometry(0.5);

no_vr.js - Let Your Scene be Seen
);
scene.add(light);

no_vr.js - Let there be light!
);
scene.add(light);

no_vr.js - Have Something to See
scene.add(light);
var bitMaterial = new THREE.MeshLambertMaterial({
color: 0x00ffff, shading: THREE.FlatShading
});
var bit = new THREE.Mesh(bitGeometry, bitMaterial);
bit.position.z = -2;
scene.add(bit);
var planeGeometry =
new THREE.PlaneBufferGeometry(1000, 1000, 1000);
var planeMaterial = new THREE.MeshPhongMaterial({

no_vr.js - Get Animated
scene.add(plane);
function animate(time) {
bit.position.x = Math.sin(time/1000) * 2;
bit.position.y = Math.sin(time/2000);
bit.rotation.y += 0.01;
bit.rotation.z += 0.01;
renderer.render(scene, camera);
requestAnimationFrame( animate );
}
animate();

WebVR is a device interface
Detect and poll Head Mounted Displays (HMDs) and
other position reporting devices.

navigator.getVRDevices()
Through callback or promise returns a list of VR devices
of two types …

PositionSensorVRDevice
Information about position and orientation

HMDVRDevice
Informs you have a surface to render on and
information about the eyes.

Math is HARD
Translating all this is difﬁcult, but luckily three.js comes
to the rescue.

Nearly all WebVR demos use
VRControls and VREffect
Found in three.js examples alongside other great
utilities.
github.com/mrdoob/three.js/tree/master/examples/js

basic_vr.html
<!DOCTYPE html>
<html lang="en">
<head>
<title>Basic VR</title>
<style>
body {
color: #fff;
margin: 0px;
padding: 0;
overflow: hidden;
}
</style>
</head>
<body>
<script src="js/VRControls.js"></script>
<script src="js/VREffect.js"></script>
<script src="js/basic_vr.js"></script>
</body>
</html>

basic_vr.html
</style>
</head>
<body>
</body>
</html>

basic_vr.js - Introduce VRControls
);
var controls = new THREE.VRControls(camera);
var effect = new THREE.VREffect(renderer);
effect.setSize(window.innerWidth, window.innerHeight);

VRControls
Handles PositionSensorDevice and manipulates the
camera orientation.

VRControl.js - Initialization
function gotVRDevices( devices )
devices = filterInvalidDevices( devices );
for ( var i = 0; i < devices.length; i ++ ) {
if ( devices[ i ] instanceof PositionSensorVRDevice ) {
vrInputs.push( devices[ i ] );
}
}
if ( onError ) onError( 'HMD not available' );
}
if ( navigator.getVRDevices ) {
navigator.getVRDevices().then( gotVRDevices );
}

basic_vr.js - Introduce VREffect
);
var controls = new THREE.VRControls(camera);
var effect = new THREE.VREffect(renderer);
effect.setSize(window.innerWidth, window.innerHeight);

VREffect
Handles HMDVRDevice and uses it to render a
stereoscopic view

VREffect.js - Initialization
function gotVRDevices( devices ) {
for ( var i = 0; i < devices.length; i ++ ) {
if ( devices[ i ] instanceof HMDVRDevice ) {
///…
}
if ( vrHMD === undefined ) {
if ( onError ) onError( 'HMD not available' );
}
}

VREffect.js - Remember the "..."
var eyeParamsL = vrHMD.getEyeParameters( 'left' );
var eyeParamsR = vrHMD.getEyeParameters( 'right' );
eyeTranslationL = eyeParamsL.eyeTranslation;
eyeTranslationR = eyeParamsR.eyeTranslation;
eyeFOVL = eyeParamsL.recommendedFieldOfView;
eyeFOVR = eyeParamsR.recommendedFieldOfView;

basic_vr.js - Go Full Screen
var vrMode = false;
function enterVR() {
effect.setFullScreen(true);
vrMode = true;
}
function exitVR() {
effect.setFullScreen(false);
vrMode = false;
};
function toggleVR() {
if (!vrMode) {
enterVR();

VREffect - Full Screen VR display
this.setFullScreen = function ( boolean ) {
if ( vrHMD === undefined ) return;
if ( isFullscreen === boolean ) return;
if ( canvas.mozRequestFullScreen ) {
canvas.mozRequestFullScreen( { vrDisplay: vrHMD } );
} else if ( canvas.webkitRequestFullscreen ) {
canvas.webkitRequestFullscreen( { vrDisplay: vrHMD } );
}
};

basic_vr.js - Let Reality Take Control
controls.update();
effect.render(scene, camera);
}
animate();

VRControls.js - Get State from VR Device
this.update = function () {
for ( var i = 0; i < vrInputs.length; i ++ ) {
var vrInput = vrInputs[ i ];
var state = vrInput.getState();
if ( state.orientation !== null ) {
object.quaternion.copy( state.orientation );
}
if ( state.position !== null ) {
object.position.copy( state.position )
.multiplyScalar( scope.scale );
}
}
};

basic_vr.js - The Full VR Effect
controls.update();
effect.render(scene, camera);
}
animate();

Two Cameras for the price of
ONE!
It creates a camera for each eye, shifts them from the
main camera location based on HMDVRDevice eye
information, then renders the view for each!

This doesn’t work on mobile
browsers!
WebVR is still in development.

webvr-polyﬁll
Supplies Mobile device gyroscope as
PositionSensorVRDevice, screen as HMDDevice.
github.com/borismus/webvr-polyﬁll

webvr-boilerplate
A very basic skeleton much like you’ve seen here today.
github.com/borismus/webvr-boilerplate

basic_vr_mobile.html - Drop It In
<!DOCTYPE html>
<html lang="en">
<head>
<title>Basic VR Mobile</title>
<style>
body {
color: #fff;
margin: 0px;
padding: 0;
overflow: hidden;
}
</style>
</head>
<body>
<script src="js/webvr-polyfill.js"></script>
</body>
</html>

basic_vr_mobile.html - Drop It In
</style>
</head>
<body>
</body>
</html>

I know what
you’re thinking …

Simulator Sickness is Real …
Need to hit 60-90 fps.
Motion data already laggy.
Additional disadvantage of being in browser.
More graphics you push the harder this gets.

Spherically map videos
around each camera/eye
Again, math is HARD!
See three.js vr_video or eleVR-Web-Player for example.

All the HTML5 Things
Keyboard, mouse, gamepad …

Leap Motion is Strongly
Embracing WebVR
Works directly with three.js
leapmotion.com/product/vr

leap_motion.html
<!DOCTYPE html>
<html lang="en">
<head>
<title>Leap Motion VR</title>
<style>
body {
color: #fff;
margin: 0px;
padding: 0;
overflow: hidden;
}
</style>
</head>
<body>
<script src="//js.leapmotion.com/leap-0.6.3.min.js"></script>
<script src="//js.leapmotion.com/leap-plugins-0.1.9.min.js"></script>
<script src="//js.leapmotion.com/leap.rigged-hand-0.1.7.min.js"></script>
<script src="js/leap_motion.js"></script>
</body>
</html>

leap_motion.html
</style>
</head>
<body>
<script src="//js.leapmotion.com/leap-0.6.3.min.js"></script>
<script src="//js.leapmotion.com/leap-plugins-0.1.9.min.js"></
script>
<script src="//js.leapmotion.com/leap.rigged-hand-0.1.7.min.js"
script>
<script src="js/leap_motion.js"></script>
</body>
</html>

leap_motion.js
Leap.loop();
Leap.loopController.use('transform', {
vr: true,
effectiveParent: camera
});
Leap.loopController.use('riggedHand', {
parent: scene,
renderer: renderer,
materialOptions: {
emissive: new THREE.Color(0x00aaaa)
}
});
animate();

This doesn’t work on mobile*
* without hacks

One interaction
to rule them all

Gaze
Where you are looking can trigger changes, from
environment to storyline.

raycast.js - Gaze
var raycaster = new THREE.Raycaster();
var center = new THREE.Vector2();
function gaze() {
if(!audioPlaying) {
raycaster.setFromCamera(center, camera);
var intersects = raycaster.intersectObjects([bit]);
if (intersects.length > 0) {
audioPlaying = true;
if (Math.random() < 0.5) {
switchBitTo(1);
yes.play();
} else {
switchBitTo(0);
no.play();

Responsive VR
For “simple” devices, it’s a roller coaster ride.
More complex, more interactions.

Math is HARD
Collision detection, physics, etc.

Unity publishes to multiple
platforms
Plugins for VR available. Support directly baked in 5.1

#pragma strict
var center : GameObject;
var axis : Vector3;
function Start () {
}
function Update () {
transform.RotateAround(
center.transform.position, axis,30 * Time.deltaTime
);
}
Orbit.js

Cores.js
function beingLookedAt() {
var hit : RaycastHit;
var ray = new Ray(Camera.main.transform.position,
Camera.main.transform.forward);
return GetComponent
.<Collider>()
.Raycast(ray, hit, Mathf.Infinity);
}

Unity now exports to WebGL
An idea …

Build a WebVR Unity plugin
Bridges when publishing to WebGL.
Design once, distribute everywhere, run anywhere.
NOW YOU’RE PLAYING WITH POWER

Again, Simulator Sickness is
Real …
It’ll get better, but start with easy motions

3D Modelling is Hard
Spent most of my time on it.
Develop interactions ﬁrst!
Then ﬁnd a friend.

The ground is constantly
changing
Beta Software running on Beta Hardware.
Things break constantly.

Thanks!
e: rudy@carbonﬁve.com
t: @rudy

Getting Started in VR with JS

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (11)

Similar to Getting Started in VR with JS

Similar to Getting Started in VR with JS (20)

Recently uploaded

Recently uploaded (20)

Getting Started in VR with JS