Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Getting Started in VR with JS

9,744 views

Published on

With Oculus, Samsung Gear, Google Cardboard, and more headsets rushing to market, it's an exciting time to enter the world of virtual reality. With frameworks from Mozilla WebVR, Unity, LeapMotion and others providing support, Javascript developers can literally get into the game.

In this talk, we'll walkthrough a simple WebVR program to see:

* the ease of getting started
* the technical, design, and UX challenges faced
* the roadmap of things to come

Published in: Technology
  • Be the first to comment

Getting Started in VR with JS

  1. 1. Getting Started in VR with JS
  2. 2. Getting Started in VR with JS The Dream of the 90s is Alive! #empirejs2015
  3. 3. MARRY ME, ALICIA!
  4. 4. GO SARNIA BEES!
  5. 5. 20 years later …
  6. 6. Why would this time be any different?
  7. 7. VR SHE WROTE
  8. 8. JavaScript FTW WebVR allows us to work in JS “native” browser environment.
  9. 9. What’s a headset? WOAH! your app renders 1. delivers position data to 2. surface to display stereoscopic image .js
  10. 10. A device that gives positional data & a surface to draw on? Sounds awful like a smart phone!
  11. 11. google.com/get/cardboard/
  12. 12. I’ll be giving these out Come see me in the Q&A lounge after this and throughout the conference.
  13. 13. Forgiveness please …
  14. 14. I AM A (DEMO) GOD HERE
  15. 15. Demos available at c5vr.com And also in the Q&A Lounge after.
  16. 16. Download ALL THE THINGS! Drivers and SDK from developer.oculus.com Firefox WebVR Browser from mozvr.com/downloads Chromium WebVR Browser from bit.ly/1DPjgDQ
  17. 17. What is WebVR?
  18. 18. What it's not!
  19. 19. Not a Virtual Reality DOM
  20. 20. Not a WebGL replacement WebGL is a framework to build your own 3D graphics rendering pipeline.
  21. 21. Math is HARD Computing per pixel transform and coloring takes a lot of work.
  22. 22. Use three.js Provides a higher level abstraction that is easier to work with.
  23. 23. c5vr.com/no_vr.html
  24. 24. no_vr.html <!DOCTYPE html> <html lang="en"> <head> <title>No VR</title> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, user-scalable=no, minimum-scale=1.0, maximum-scale=1.0"> <meta name="mobile-web-app-capable" content="yes"> <meta name="apple-mobile-web-app-capable" content="yes" /> <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" /> <style> body { background-color: #000; color: #fff; margin: 0px; padding: 0; overflow: hidden; } </style> </head> <body> <script src="js/three.js"></script> <script src="js/no_vr.js"></script> </body> </html>
  25. 25. no_vr.html </style> </head> <body> <script src="js/three.js"></script> <script src="js/no_vr.js"></script> </body> </html>
  26. 26. no_vr.js - Create a Renderer var renderer = new THREE.WebGLRenderer({antialias: true}); renderer.setPixelRatio(window.devicePixelRatio); renderer.setSize(window.innerWidth, window.innerHeight); document.body.appendChild(renderer.domElement); var scene = new THREE.Scene(); var camera = new THREE.PerspectiveCamera( 75, window.innerWidth / window.innerHeight, 0.3, 10000 ); var light = new THREE.PointLight(0xffffff, 1.0, 0); light.position.set(0,0,0); scene.add(light); var bitGeometry = new THREE.DodecahedronGeometry(0.5);
  27. 27. no_vr.js - Let Your Scene be Seen var renderer = new THREE.WebGLRenderer({antialias: true}); renderer.setPixelRatio(window.devicePixelRatio); renderer.setSize(window.innerWidth, window.innerHeight); document.body.appendChild(renderer.domElement); var scene = new THREE.Scene(); var camera = new THREE.PerspectiveCamera( 75, window.innerWidth / window.innerHeight, 0.3, 10000 ); var light = new THREE.PointLight(0xffffff, 1.0, 0); light.position.set(0,0,0); scene.add(light); var bitGeometry = new THREE.DodecahedronGeometry(0.5);
  28. 28. no_vr.js - Let there be light! var renderer = new THREE.WebGLRenderer({antialias: true}); renderer.setPixelRatio(window.devicePixelRatio); renderer.setSize(window.innerWidth, window.innerHeight); document.body.appendChild(renderer.domElement); var scene = new THREE.Scene(); var camera = new THREE.PerspectiveCamera( 75, window.innerWidth / window.innerHeight, 0.3, 10000 ); var light = new THREE.PointLight(0xffffff, 1.0, 0); light.position.set(0,0,0); scene.add(light); var bitGeometry = new THREE.DodecahedronGeometry(0.5);
  29. 29. no_vr.js - Have Something to See var light = new THREE.PointLight(0xffffff, 1.0, 0); light.position.set(0,0,0); scene.add(light); var bitGeometry = new THREE.DodecahedronGeometry(0.5); var bitMaterial = new THREE.MeshLambertMaterial({ color: 0x00ffff, shading: THREE.FlatShading }); var bit = new THREE.Mesh(bitGeometry, bitMaterial); bit.position.z = -2; scene.add(bit); var planeGeometry = new THREE.PlaneBufferGeometry(1000, 1000, 1000); var planeMaterial = new THREE.MeshPhongMaterial({
  30. 30. no_vr.js - Get Animated scene.add(plane); function animate(time) { bit.position.x = Math.sin(time/1000) * 2; bit.position.y = Math.sin(time/2000); bit.rotation.y += 0.01; bit.rotation.z += 0.01; renderer.render(scene, camera); requestAnimationFrame( animate ); } animate();
  31. 31. no_vr.js - Get Animated scene.add(plane); function animate(time) { bit.position.x = Math.sin(time/1000) * 2; bit.position.y = Math.sin(time/2000); bit.rotation.y += 0.01; bit.rotation.z += 0.01; renderer.render(scene, camera); requestAnimationFrame( animate ); } animate();
  32. 32. no_vr.js - Get Animated scene.add(plane); function animate(time) { bit.position.x = Math.sin(time/1000) * 2; bit.position.y = Math.sin(time/2000); bit.rotation.y += 0.01; bit.rotation.z += 0.01; renderer.render(scene, camera); requestAnimationFrame( animate ); } animate();
  33. 33. no_vr.js - Get Animated scene.add(plane); function animate(time) { bit.position.x = Math.sin(time/1000) * 2; bit.position.y = Math.sin(time/2000); bit.rotation.y += 0.01; bit.rotation.z += 0.01; renderer.render(scene, camera); requestAnimationFrame( animate ); } animate();
  34. 34. no_vr.js - Get Animated scene.add(plane); function animate(time) { bit.position.x = Math.sin(time/1000) * 2; bit.position.y = Math.sin(time/2000); bit.rotation.y += 0.01; bit.rotation.z += 0.01; renderer.render(scene, camera); requestAnimationFrame( animate ); } animate();
  35. 35. Apply some WebVR and …
  36. 36. c5vr.com/basic_vr.html
  37. 37. What is WebVR?
  38. 38. WebVR is a device interface Detect and poll Head Mounted Displays (HMDs) and other position reporting devices.
  39. 39. navigator.getVRDevices() Through callback or promise returns a list of VR devices of two types …
  40. 40. PositionSensorVRDevice Information about position and orientation
  41. 41. HMDVRDevice Informs you have a surface to render on and information about the eyes.
  42. 42. Math is HARD Translating all this is difficult, but luckily three.js comes to the rescue.
  43. 43. Nearly all WebVR demos use VRControls and VREffect Found in three.js examples alongside other great utilities. github.com/mrdoob/three.js/tree/master/examples/js
  44. 44. basic_vr.html <!DOCTYPE html> <html lang="en"> <head> <title>Basic VR</title> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, user-scalable=no, minimum-scale=1.0, maximum-scale=1.0"> <meta name="mobile-web-app-capable" content="yes"> <meta name="apple-mobile-web-app-capable" content="yes" /> <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" /> <style> body { background-color: #000; color: #fff; margin: 0px; padding: 0; overflow: hidden; } </style> </head> <body> <script src="js/three.js"></script> <script src="js/VRControls.js"></script> <script src="js/VREffect.js"></script> <script src="js/basic_vr.js"></script> </body> </html>
  45. 45. basic_vr.html </style> </head> <body> <script src="js/three.js"></script> <script src="js/VRControls.js"></script> <script src="js/VREffect.js"></script> <script src="js/basic_vr.js"></script> </body> </html>
  46. 46. basic_vr.js - Introduce VRControls var renderer = new THREE.WebGLRenderer({antialias: true}); renderer.setPixelRatio(window.devicePixelRatio); document.body.appendChild(renderer.domElement); var scene = new THREE.Scene(); var camera = new THREE.PerspectiveCamera( 75, window.innerWidth / window.innerHeight, 0.3, 10000 ); var controls = new THREE.VRControls(camera); var effect = new THREE.VREffect(renderer); effect.setSize(window.innerWidth, window.innerHeight); var light = new THREE.PointLight(0xffffff, 1.0, 15); light.position.set(0,0,0);
  47. 47. VRControls Handles PositionSensorDevice and manipulates the camera orientation.
  48. 48. VRControl.js - Initialization function gotVRDevices( devices ) devices = filterInvalidDevices( devices ); for ( var i = 0; i < devices.length; i ++ ) { if ( devices[ i ] instanceof PositionSensorVRDevice ) { vrInputs.push( devices[ i ] ); } } if ( onError ) onError( 'HMD not available' ); } if ( navigator.getVRDevices ) { navigator.getVRDevices().then( gotVRDevices ); }
  49. 49. basic_vr.js - Introduce VREffect var renderer = new THREE.WebGLRenderer({antialias: true}); renderer.setPixelRatio(window.devicePixelRatio); document.body.appendChild(renderer.domElement); var scene = new THREE.Scene(); var camera = new THREE.PerspectiveCamera( 75, window.innerWidth / window.innerHeight, 0.3, 10000 ); var controls = new THREE.VRControls(camera); var effect = new THREE.VREffect(renderer); effect.setSize(window.innerWidth, window.innerHeight); var light = new THREE.PointLight(0xffffff, 1.0, 15); light.position.set(0,0,0);
  50. 50. VREffect Handles HMDVRDevice and uses it to render a stereoscopic view
  51. 51. VREffect.js - Initialization function gotVRDevices( devices ) { for ( var i = 0; i < devices.length; i ++ ) { if ( devices[ i ] instanceof HMDVRDevice ) { ///… } if ( vrHMD === undefined ) { if ( onError ) onError( 'HMD not available' ); } }
  52. 52. VREffect.js - Remember the "..." var eyeParamsL = vrHMD.getEyeParameters( 'left' ); var eyeParamsR = vrHMD.getEyeParameters( 'right' ); eyeTranslationL = eyeParamsL.eyeTranslation; eyeTranslationR = eyeParamsR.eyeTranslation; eyeFOVL = eyeParamsL.recommendedFieldOfView; eyeFOVR = eyeParamsR.recommendedFieldOfView;
  53. 53. basic_vr.js - Go Full Screen var vrMode = false; function enterVR() { effect.setFullScreen(true); vrMode = true; } function exitVR() { effect.setFullScreen(false); vrMode = false; }; function toggleVR() { if (!vrMode) { enterVR();
  54. 54. VREffect - Full Screen VR display this.setFullScreen = function ( boolean ) { if ( vrHMD === undefined ) return; if ( isFullscreen === boolean ) return; if ( canvas.mozRequestFullScreen ) { canvas.mozRequestFullScreen( { vrDisplay: vrHMD } ); } else if ( canvas.webkitRequestFullscreen ) { canvas.webkitRequestFullscreen( { vrDisplay: vrHMD } ); } };
  55. 55. basic_vr.js - Let Reality Take Control function animate(time) { bit.position.x = Math.sin(time/1000) * 2; bit.position.y = Math.sin(time/2000); bit.rotation.y += 0.01; bit.rotation.z += 0.01; controls.update(); effect.render(scene, camera); requestAnimationFrame( animate ); } animate();
  56. 56. VRControls.js - Get State from VR Device this.update = function () { for ( var i = 0; i < vrInputs.length; i ++ ) { var vrInput = vrInputs[ i ]; var state = vrInput.getState(); if ( state.orientation !== null ) { object.quaternion.copy( state.orientation ); } if ( state.position !== null ) { object.position.copy( state.position ) .multiplyScalar( scope.scale ); } } };
  57. 57. basic_vr.js - The Full VR Effect function animate(time) { bit.position.x = Math.sin(time/1000) * 2; bit.position.y = Math.sin(time/2000); bit.rotation.y += 0.01; bit.rotation.z += 0.01; controls.update(); effect.render(scene, camera); requestAnimationFrame( animate ); } animate();
  58. 58. Two Cameras for the price of ONE! It creates a camera for each eye, shifts them from the main camera location based on HMDVRDevice eye information, then renders the view for each!
  59. 59. Confession
  60. 60. This doesn’t work on mobile browsers! WebVR is still in development.
  61. 61. webvr-polyfill Supplies Mobile device gyroscope as PositionSensorVRDevice, screen as HMDDevice. github.com/borismus/webvr-polyfill
  62. 62. webvr-boilerplate A very basic skeleton much like you’ve seen here today. github.com/borismus/webvr-boilerplate
  63. 63. basic_vr_mobile.html - Drop It In <!DOCTYPE html> <html lang="en"> <head> <title>Basic VR Mobile</title> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, user-scalable=no, minimum-scale=1.0, maximum-scale=1.0"> <meta name="mobile-web-app-capable" content="yes"> <meta name="apple-mobile-web-app-capable" content="yes" /> <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" /> <style> body { background-color: #000; color: #fff; margin: 0px; padding: 0; overflow: hidden; } </style> </head> <body> <script src="js/three.js"></script> <script src="js/VRControls.js"></script> <script src="js/VREffect.js"></script> <script src="js/webvr-polyfill.js"></script> <script src="js/basic_vr.js"></script> </body> </html>
  64. 64. basic_vr_mobile.html - Drop It In </style> </head> <body> <script src="js/three.js"></script> <script src="js/VRControls.js"></script> <script src="js/VREffect.js"></script> <script src="js/webvr-polyfill.js"></script> <script src="js/basic_vr.js"></script> </body> </html>
  65. 65. I know what you’re thinking …
  66. 66. Simulator Sickness is Real … Need to hit 60-90 fps. Motion data already laggy. Additional disadvantage of being in browser. More graphics you push the harder this gets.
  67. 67. Consider 360 Video
  68. 68. Spherically map videos around each camera/eye Again, math is HARD! See three.js vr_video or eleVR-Web-Player for example.
  69. 69. Interaction
  70. 70. All the HTML5 Things Keyboard, mouse, gamepad …
  71. 71. Leap Motion is Strongly Embracing WebVR Works directly with three.js leapmotion.com/product/vr
  72. 72. c5vr.com/leap_motion.html
  73. 73. leap_motion.html <!DOCTYPE html> <html lang="en"> <head> <title>Leap Motion VR</title> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, user-scalable=no, minimum-scale=1.0, maximum-scale=1.0"> <meta name="mobile-web-app-capable" content="yes"> <meta name="apple-mobile-web-app-capable" content="yes" /> <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" /> <style> body { background-color: #000; color: #fff; margin: 0px; padding: 0; overflow: hidden; } </style> </head> <body> <script src="js/three.js"></script> <script src="js/VRControls.js"></script> <script src="js/VREffect.js"></script> <script src="js/webvr-polyfill.js"></script> <script src="//js.leapmotion.com/leap-0.6.3.min.js"></script> <script src="//js.leapmotion.com/leap-plugins-0.1.9.min.js"></script> <script src="//js.leapmotion.com/leap.rigged-hand-0.1.7.min.js"></script> <script src="js/leap_motion.js"></script> </body> </html>
  74. 74. leap_motion.html </style> </head> <body> <script src="js/three.js"></script> <script src="js/VRControls.js"></script> <script src="js/VREffect.js"></script> <script src="js/webvr-polyfill.js"></script> <script src="//js.leapmotion.com/leap-0.6.3.min.js"></script> <script src="//js.leapmotion.com/leap-plugins-0.1.9.min.js"></ script> <script src="//js.leapmotion.com/leap.rigged-hand-0.1.7.min.js" script> <script src="js/leap_motion.js"></script> </body> </html>
  75. 75. leap_motion.js Leap.loop(); Leap.loopController.use('transform', { vr: true, effectiveParent: camera }); Leap.loopController.use('riggedHand', { parent: scene, renderer: renderer, materialOptions: { emissive: new THREE.Color(0x00aaaa) } }); animate();
  76. 76. Confession 2.0
  77. 77. This doesn’t work on mobile* * without hacks
  78. 78. One interaction to rule them all
  79. 79. Gaze Where you are looking can trigger changes, from environment to storyline.
  80. 80. c5vr.com/raycast_vr.html
  81. 81. raycast.js - Gaze var raycaster = new THREE.Raycaster(); var center = new THREE.Vector2(); function gaze() { if(!audioPlaying) { raycaster.setFromCamera(center, camera); var intersects = raycaster.intersectObjects([bit]); if (intersects.length > 0) { audioPlaying = true; if (Math.random() < 0.5) { switchBitTo(1); yes.play(); } else { switchBitTo(0); no.play();
  82. 82. Responsive VR For “simple” devices, it’s a roller coaster ride. More complex, more interactions.
  83. 83. Math is HARD Collision detection, physics, etc.
  84. 84. Unity publishes to multiple platforms Plugins for VR available. Support directly baked in 5.1
  85. 85. #pragma strict var center : GameObject; var axis : Vector3; function Start () { } function Update () { transform.RotateAround( center.transform.position, axis,30 * Time.deltaTime ); } Orbit.js
  86. 86. Cores.js function beingLookedAt() { var hit : RaycastHit; var ray = new Ray(Camera.main.transform.position, Camera.main.transform.forward); return GetComponent .<Collider>() .Raycast(ray, hit, Mathf.Infinity); }
  87. 87. Unity now exports to WebGL An idea …
  88. 88. Build a WebVR Unity plugin Bridges when publishing to WebGL. Design once, distribute everywhere, run anywhere. NOW YOU’RE PLAYING WITH POWER
  89. 89. Gotchas
  90. 90. Again, Simulator Sickness is Real … It’ll get better, but start with easy motions
  91. 91. 3D Modelling is Hard Spent most of my time on it. Develop interactions first! Then find a friend.
  92. 92. The ground is constantly changing Beta Software running on Beta Hardware. Things break constantly.
  93. 93. So that’s it …
  94. 94. Except for one thing …?
  95. 95. Why would this time be any different?
  96. 96. Answer: Because of you …
  97. 97. Thanks! e: rudy@carbonfive.com t: @rudy

×