Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

VR Everywhere March 2015


Published on

My introductory VR talk at NVIDIA GTC 2015

Published in: Technology

VR Everywhere March 2015

  1. 1. VR everywhere consumer virtual reality for desktop, mobile and web tony parisi march, 2015
  2. 2. about me 3/18/2015 CONTACT skype: auradeluxe http://www.tonypa GET THE BOOKS! WebGL: Up and Running Programming 3D Applications with HTML and WebGL Visualization/dp/1449362966 MEETUPS BOOK CODE GET GLAM WORK CREDS Co-creator, VRML and X3D
  3. 3. the promised land! consumer VR 3/18/2015
  4. 4. hardware explosion 3/18/2015 headsets input devices high performance laptops AFFORDABLE STEREOSCOPIC DISPLAYS, CRAZY-FAST GPUS, CHEAP MOTION TRACKING IMUS… FTW!
  5. 5. PC HMDs 3/18/2015  Provide deepest, most immersive VR experiences  Desktop peripheral - acts as external monitor  Position and orientation tracking  Tethered – one or more cables to the computer; camera for position tracking  Oculus Rift – the Gold Standard of VR headsets  Developer Kit CHEAP $350  In production late 2015/early 2016  HTC/Valve VIVE  OSVR – Razer open dev kit and SDK; ecosystem initiative
  6. 6. mobile HMDs 3/18/2015  Deluxe Mobile – Samsung GearVR  Optics and IMU licensed from Oculus  Custom Android build w/ VR extensions and enhanced UI  The best VR experience out there! (Look Ma, no wires)  BUT $200 + requires new Phone (Note 4) NOT CHEAP $1,000  Support for S6 coming, price coming down  Orientation tracking only  So far… no competition… but EXPECT IT!
  7. 7. drop-in phone viewers 3/18/2015  Google Cardboard  Just Add SmartPhone  Orientation tracking only  Open specification for mobile VR  Standard Android OS support, using simple stereo rendering and accelerometer tracking  $2 in parts or $25 ready-to-assemble Kits WAY CHEAP  Not as nice… but great for the price!  MergeVR, Wearality, Moggles…  Some of these have different rendering thus require custom SDK
  8. 8. operating systems 3/18/2015  Desktop – Oculus SDK supports Windows/Mac/Linux; Vive SDK not yet released…  GearVR - custom Android build w/Oculus mobile SDK  Cardboard  Google has SDK for native Android with simple stereo render and head tracking; also adding first-class VR support into future OS  Cardboard is just side-by side rendering and accelerometer tracking… so it works on iOS too  Mobile browsers can run Cardboard apps! Just stereo render with WebGL and track w/DeviceOrientation API The Oculus Store for GearVR
  9. 9. rendering 3/18/2015 Oculus Rift/GearVR  110 diagonal FOV  Side-by-side stereo  Barrel distortion Cardboard  90 degree FOV  Side-by-side stereo
  10. 10. head tracking 3/18/2015  Oculus Rift/GearVR  1000hz refresh rate  Cardboard  200hz refresh rate  Desktop IMUs built into HMDs track position and orientation. Use camera or wall-mounted sensors for position tracking  Mobile IMUs track orientation only INERTIAL MEASURE UNITS (IMUS)
  11. 11. the challenge: “presence” 3/18/2015  Camera must respond to head tracking at 75FPS+  higher latency risks motion sickness and breaks the illusion  Rendering, as always, must be at 60FPS+  “Motions-to-photons” latency goal is <20ms  Desktop and GearVR can hit this, currently mobile can’t (though it’s in the works); most phone IMU motions-to-photons are are 50ms+  What’s wrong with this picture???  Classic targeting of 60FPS makes rendering the bottleneck  If targeting 75FPS, that’s minimum 150FPS simulation (75FPS x 2 cameras), or <7ms frame time!!!
  12. 12. performance (1) 3/18/2015  Software techniques  Of course push everything you can onto the GPU  New engine optimizations Example: render each state change to each camera vs. whole scene to each camera  Mobile: create experiences with lower res than typical games... till the hardware gets faster and resolutions get higher; low latency trumps hi poly count  Time Warp  Oculus SDK technique, transforms image in Z-buffer at next refresh based on new IMU position/orientation… even if new rendered frame isn’t ready  Already supported in game engines and other runtimes
  13. 13. performance (2) 3/18/2015 First attempts at a comprehensive GPU approach to VR - NVIDIA VR Direct  New features enabled in Maxwell processors (GeForce GTX 980 and GTX 970)  Overall attack on latency – whole pipeline  Asynchronous Warp (NVIDIA’s version of Time Warp)  Dynamic Super Resolution (DSR) – up to 4 megapixels per eye downsampled to HD  Multi-frame sampled anti-aliasing (MFAA)  Boasts motion-to-photons latency of 25ms  … plus “GeForce Experience” control panel – one end-user settings dialog instead of several ACCELERATING VR USING THE POWER OF THE GPU
  14. 14. developing desktop and mobile VR 3/18/2015  Game engines  Unity3D – targets all platforms: desktop Oculus, Gear VR, Cardboard. Free and open prefabs available on Unity store and from HMD vendors. Expect Vive and other platforms quickly; they move fast.  Unreal - desktop now, mobile in development  Native SDKs  Oculus SDK C++  Oculus Mobile SDK C++  Android SDK for Cardboard
  15. 15. WebVR: fast, cheap and democratized 3/18/2015  Experimental browser API  Query browser for attached HMD devices  Render fullscreen stereo in WebGL  Track HMD position/orientation by polling HMD device  Already in Firefox and Chrome nightly builds, uses same API  Build apps using open source JavaScript engines, or WebGL “exported” from Unity/Unreal via Emscripten cross-compiler  Browser refresh currently throttled at 60FPS but they are lifting this limitation Quake 3 WebVR demo, developed by Brandon Jones of Google NO-DOWNLOAD VR APPLICATIONS. INSTANT ACCESS IN THE BROWSER, SHARE IT WITH A URL! WRITTEN IN JAVASCRIPT!?
  16. 16. input 3/18/2015  We still don’t have the mouse of VR  State of the practice is blind use of keyboard or gamepad controller  Motion controllers like Leap Motion, Sixense STEM system are gaining popularity for PC-based VR  GearVR has a really innovative trackpad design, plus simple, elegant navigation UI  Drop-in phone viewers are experimenting with side-mounted click devices and Bluetooth controllers Leap Motion controller Sixense STEM System image:
  17. 17. augmented reality 3/18/2015  Explosion of VR has renewed interest in AR  Promising new hardware  Microsoft Hololens  CastAR - table-top AR display for gaming, close to coming to market  Magic Leap ????  A little farther out… most projects don’t have ship dates  Another SDK babel looming… THE NEXT NEXT BIG THING? Hololens Sculpting Tool Demo CastAR Multi-Player Game
  18. 18. coming in May 2015 3/18/2015
  19. 19. keep in touch 3/18/2015 CONTACT skype: auradeluxe http://www.tonypa MEETUPS BOOK CODE GET GLAM WORK CREDS Co-creator, VRML and X3D GET THE BOOKS! WebGL: Up and Running Programming 3D Applications with HTML and WebGL Visualization/dp/1449362966
  20. 20. VR everywhere consumer virtual reality for desktop, mobile and web tony parisi march, 2015