Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Hacking Reality: Browser-Based VR with HTML5

7,676 views

Published on

Slides from fall 2014 HTML5 Dev Conf talk on Browser-Based VR

Published in: Technology

Hacking Reality: Browser-Based VR with HTML5

  1. 1. HACKING REALITY: HTML5 AND JAVASCRIPT FOR CROSS-PLATFORM VR TONY PARISI OCTOBER, 2014
  2. 2. ABOUT ME CREDS Co-creator, VRML and X3D http://www.tonyparisi.com 10/21/2014 CONTACT tony@vizi.gl skype: auradeluxe http://twitter.com/auradeluxe http://www.tonyparisi.com/ http://www.learningwebgl.com/ GET VIZI https://github.com/tparisi/Vizi GET THE BOOKS! WebGL: Up and Running http://www.amazon.com/dp/144932357X Programming 3D Applications with HTML and WebGL http://www.amazon.com/Programming-Applications-HTML5-WebGL-Visualization/ dp/1449362966 MEETUPS http://www.meetup.com/WebGL-Developers-Meetup/ http://www.meetup.com/Web-VR/ BOOK CODE https://github.com/tparisi/WebGLBook https://github.com/tparisi/Programming3DApplications GET GLAM http://www.glamjs.org/ https://github.com/tparisi/glam/ WORK http://www.vizi.gl/
  3. 3. THE PROMISED LAND! CONSUMER VR http://www.tonyparisi.com 10/21/2014
  4. 4. VR TODAY http://www.tonyparisi.com 10/21/2014 GIANT DOWNLOADS SILO EXPERIENCES CUMBERSOME NATIVE APPS AND INSTALLS PROPRIETARY PLATFORMS AND DEVELOPMENT STACKS VR DOTH BE HARD!
  5. 5. WHY I LOVE THE WEB  INSTANT PUBLISHING  INSTANT ACCESS TO INFORMATION  NO TOLLS  NOBODY CONTROLS IT  CULTURE OF COLLABORATION  VIEW SOURCE …THE WEB WILL NEVER CLOSE UP SHOP. image: Mark Surman http://commonspace.wordpress.com/2014/03/12/happybirthday/ http://www.tonyparisi.com 10/21/2014
  6. 6. THE THREE D’S OF THE WEB http://www.tonyparisi.com 10/21/2014  DEVELOPMENT  CROSS-PLATFORM  VENDOR-NEUTRAL  OPEN SOURCE  DEPLOYMENT  CLOUD  PUSH-BUTTON UPDATE AND PUBLISH  DISTRIBUTION AND DISCOVERY  EMBED IN OTHER PAGES  SHARE WITH A HYPERLINK  NO APP TO DOWNLOAD
  7. 7. VR AND THE WEB: TWO GREAT TASTES… ? http://www.tonyparisi.com 10/21/2014
  8. 8. WEB VR FAST, CHEAP, AND TOTALLY DEMOCRATIZED.  BROWSER-BASED VIRTUAL REALITY  WEBGL  CSS3  VR SUPPORT NOW IN BROWSER DEV BUILDS!!!  NO BIG APP DOWNLOADS AND INSTALLS!!! http://www.tonyparisi.com 10/21/2014  JUST ADD SMART PHONE  “SMARTVR” USING GOOGLE CARDBOARD $25 CHEAP!
  9. 9. AN EXAMPLE http://www.tonyparisi.com 10/21/2014 INFORMATION DIVING SHOWCASE http://mozvr.github.io/infodive/ POWERED BY FIREFOX BUILT WITH VIZI https://github.com/tparisi/Vizi
  10. 10. THE WEBVR API 1. QUERY FOR VR DEVICE(S) TO RENDER // polyfill var getVRDevices = navigator.mozGetVRDevices /* FF */ || navigator.getVRDevices; /* webkit */ http://www.tonyparisi.com 10/21/2014 var self = this; getVRDevices().then( gotVRDevices ); function gotVRDevices( devices ) { var vrHMD; var error; for ( var i = 0; i < devices.length; ++i ) { if ( devices[i] instanceof HMDVRDevice ) { vrHMD = devices[i]; self._vrHMD = vrHMD; self.leftEyeTranslation = vrHMD.getEyeTranslation( "left" ); self.rightEyeTranslation = vrHMD.getEyeTranslation( "right" ); self.leftEyeFOV = vrHMD.getRecommendedEyeFieldOfView( "left" ); self.rightEyeFOV = vrHMD.getRecommendedEyeFieldOfView( "right" ); break; // We keep the first we encounter } } } get left/right eye (camera) positions get per-eye camera field of view; use WebGL to render scene from two cameras
  11. 11. THE WEBVR API 2. GO FULLSCREEN (VR MODE) fullscreen mode requires a DOM element http://www.tonyparisi.com 10/21/2014 var self = this; var renderer = this._renderer; var vrHMD = this._vrHMD; var canvas = renderer.domElement; var fullScreenChange = canvas.mozRequestFullScreen? 'mozfullscreenchange' : 'webkitfullscreenchange'; document.addEventListener( fullScreenChange, onFullScreenChanged, false ); function onFullScreenChanged() { if ( !document.mozFullScreenElement && !document.webkitFullScreenElement ) { self.setFullScreen( false ); } } if ( canvas.mozRequestFullScreen ) { canvas.mozRequestFullScreen( { vrDisplay: vrHMD } ); } else { canvas.webkitRequestFullscreen( { vrDisplay: vrHMD } ); } handle exiting fullscreen mode request fullscreen mode for this VR device
  12. 12. THE WEBVR API 3. HEAD TRACKING query HMD device state http://www.tonyparisi.com 10/21/2014 // polyfill var getVRDevices = navigator.mozGetVRDevices /* FF */ || navigator.getVRDevices; /* webkit */ var self = this; getVRDevices().then( gotVRDevices ); function gotVRDevices( devices ) { var vrInput; var error; for ( var i = 0; i < devices.length; ++i ) { if ( devices[i] instanceof PositionSensorVRDevice ) { vrInput = devices[i] self._vrInput = vrInput; break; // We keep the first we encounter } } } … // called once per tick from requestAnimationFrame() var update = function() { var vrState = this.getVRState(); if ( !vrState ) { return; } // vrState.hmd.rotation contains four floats, quaternion x,y,z,w setCameraRotation(vrState.hmd.rotation); }; get head-tracking VR device update camera to match HMD device orientation
  13. 13. WEBVR AND CARDBOARD  GOOGLE CARDBOARD SHOWCASE  Mobile Chrome http://g.co/chromevr  Desktop Chrome http://vr.chromeexperiments.com/  EVEN EASIER  RENDER WEBGL SIDE-BY-SIDE STEREO (NO NEED TO QUERY DEVICES)  USE BROWSER DEVICE ORIENTATION API FOR HEAD TRACKING http://www.tonyparisi.com 10/21/2014
  14. 14. WEBVR AND THREE.JS  THE MOST POPULAR WEBGL LIBRARY  http://threejs.org/  LATEST STABLE VERSION (r68) INCLUDES STEREO RENDERING AND HEAD TRACKING  RENDERING examples/js/effects/StereoEffect.js (Cardboard) examples/js/effects/VREffect.js (Rift)  HEAD TRACKING examples/js/controls/DeviceOrientationControls.js (Cardboard) examples/js/controls/VRControls.js (Rift) http://www.tonyparisi.com 10/21/2014
  15. 15. VIZI: A FRAMEWORK FOR WEBVR APPLICATIONS  GOAL: MAKE IT EASY TO QUICKLY BUILD INTERESTING 3D APPLICATIONS  ARCHITECTURE INSPIRED INSPIRED BY MODERN GAME ENGINE DESIGN  COMPONENT-BASED DESIGN – EASILY PLUG NEW FEATURES INTO OBJECTS  APPLICATION OBJECT – HANDLES EVENT LOOP, THREE.JS CREATION, PAGE RESIZE, ROUTING EVENTS TO OBJECTS  INTERACTIONS – MAKE IT EASY TO PUT MOUSE AND TOUCH INTERACTION ON OBJECTS  BEHAVIORS – PREBUILT BEHAVIORS AUTOMATICALLY ROTATE, MOVE ETC.  PRE-BUILT OBJECTS – FOR COMMONLY USED DESIGN PATTERNS  SIMILAR IN DESIGN TO UNITY3D  USES THREE.JS FOR ALL GRAPHICS  ENHANCES THREE.JS VR RENDERING AND CONTROLLERS http://www.tonyparisi.com 10/21/2014
  16. 16. OPEN TOOLS FOR CROSS-PLATFORM VR http://www.tonyparisi.com 10/21/2014 game engines/IDEs Goo Enginehttp://www.gootechnologies.com/ Verold http://verold.com/ Turbulenz https://turbulenz.com/ PlayCanvas http://www.playcanvas.com/ Artillery Engine https://artillery.com/ Sketchfab https://sketchfab.com/ Unreal https://www.unrealengine.com/ Unity http://unity3d.com/#unity-5 scene graph libraries/page frameworks Three.js http://threejs.org/ SceneJS http://scenejs.org/ BabylonJS http://www.babylonjs.com/ Vizi https://github.com/tparisi/Vizi Voodoo.js http://www.voodoojs.com/ PhiloGL http://www.senchalabs.org/philogl/ tQuery http://jeromeetienne.github.io/tquery/
  17. 17. PRO TOOLS FOR WEB VR Unreal 4 in WebGL https://www.youtube.com/watch?v=c2uNDlP4RiE#t=42 60FPS ported in 5 days Unreal native C++ engine -> JavaScript Emscripten + asm.js http://www.tonyparisi.com 10/21/2014 EMSCRIPTEN - THE COOLEST HACK EVER! https://github.com/kripken/ems cripten  CROSS-COMPILE C++ NATIVE CODE TO JAVASCRIPT  asm.js- LOW-LEVEL OPTIMIZED JAVASCRIPT  UNITY, UNREAL ENGINES USE THIS TO DEPLOY ON THE WEB  WATCH OUT: HUGE DOWNLOADS AND HARD TO DEBUG…. !
  18. 18. WEBVR AND CSS http://www.tonyparisi.com 10/21/2014 MOZILLA VR CSS SHOWCASE http://mozvr.github.io/vr-web-examples/domvr-birds/
  19. 19. WEBVR AND CSS http://www.tonyparisi.com 10/21/2014 <script type="text/javascript" src="js/vrutils.js"></script> <script> /* VR Headset and its associated orientation/position sensor */ var vrHMD, vrSensor; /* Element that will serve as our camera, moving the rest of the scene around */ var cssCamera = document.getElementById("camera"); /* the camera's position, as a css transform string. For right now, we want it just in the middle. */ var cssCameraPositionTransform = "translate3d(0, 0, 0)"; /* Request animation frame loop. */ function animate() { /* Acquire positional data from VR headset and convert into a transformation that can be applied to CSS. */ if (vrSensor !== undefined) { var state = vrSensor.getState(); var cssOrientationMatrix = cssMatrixFromOrientation(state.orientation, true); } /* Apply positional data to camera element */ cssCamera.style.transform = cssOrientationMatrix + " " + cssCameraPositionTransform; window.requestAnimationFrame(animate); } query HMD device state calculate “camera” orientation update “camera’s” CSS 3D transform
  20. 20. VR + ML A MARKUP LANGUAGE FOR THE METAVERSE? http://www.tonyparisi.com 10/21/2014  GLAM (GL AND MARKUP) - A DECLARATIVE LANGUAGE FOR 3D WEB CONTENT https://github.com/tparisi/glam/  DEFINE 3D SCENE CONTENT IN MARKUP; STYLE IT IN CSS THE MARKUP <glam> <renderer type="rift"></renderer> <scene> <controller type="rift"></controller> <background id="sb1" class="skybox"> </background> <group y ='1' z='-3'> <sphere class="bubble skybox”> </sphere> <sphere class="bubble skybox”> </sphere> </group> … THE CSS <style> .skybox { envmap-right:url(../images/Park2/posx.jpg); … } .bubble { radius:.5; shader-vertex:url(../shaders/fresnel.vs); shader-fragment:url(../shaders/fresnel.fs); shader-uniforms:mRefractionRatio f 1.02 mFresnelBias f 0.1 mFresnelPower f 2.0 mFresnelScale f 1.0 tCube t null; } #sb1 { background-type:box; } </style>
  21. 21. CHALLENGES  WEBVR ON OCULUS IS NOT READY FOR PRIME TIME  (THAT’S OK NEITHER IS OCULUS!)  LATENCY IS THE BIGGEST ISSUE – BROWSER NEEDS TO UN-THROTTLE AT 60FPS  BUT WE’RE GOOD TO GO ON CARDBOARD!  60FPS WORKS GREAT  NEARLY 2 BILLION VR DEVICES ALREADY DEPLOYED!  FRAMEWORKS AND TOOLS ARE TYPICALLY WEB-ISH: UNDER DEVELOPMENT, ALWAYS IN FLUX, PRETTY MUCH OUT OF CONTROL  BUT OPEN SOURCE SO WE CAN LIVE WITH IT http://www.tonyparisi.com 10/21/2014
  22. 22. LINKS  BROWSER DEV BUILDS Firefox http://blog.bitops.com/blog/2014/08/20/updated-firefox-vr-builds/ Chrome https://drive.google.com/folderview?id=0BzudLt22BqGRbW9WTHMtOWMzNjQ  MOZILLA NEXT STEPS FOR VR (THURSDAY 23 OCT 2014) https://air.mozilla.org/virtual-reality-the-web-next-steps/  SAN FRANCISCO WEBVR MEETUP http://www.meetup.com/Web-VR/  WEBVR MAILING LIST web-vr-discuss@mozilla.org  CARDBOARD VR SHOWCASE http://vr.chromeexperiments.com/ http://www.tonyparisi.com 10/21/2014
  23. 23. KEEP IN TOUCH CREDS Co-creator, VRML and X3D http://www.tonyparisi.com 10/21/2014 CONTACT tony@vizi.gl skype: auradeluxe http://twitter.com/auradeluxe http://www.tonyparisi.com/ http://www.learningwebgl.com/ GET VIZI https://github.com/tparisi/Vizi GET THE BOOKS! WebGL: Up and Running http://www.amazon.com/dp/144932357X Programming 3D Applications with HTML and WebGL http://www.amazon.com/Programming-Applications-HTML5-WebGL-Visualization/ dp/1449362966 MEETUPS http://www.meetup.com/WebGL-Developers-Meetup/ http://www.meetup.com/Web-VR/ BOOK CODE https://github.com/tparisi/WebGLBook https://github.com/tparisi/Programming3DApplications GET GLAM http://www.glamjs.org/ https://github.com/tparisi/glam/ WORK http://www.vizi.gl/
  24. 24. HACKING REALITY: HTML5 AND JAVASCRIPT FOR CROSS-PLATFORM VR TONY PARISI OCTOBER, 2014

×