SAE AR/VR - The challenges of creating a VR application with Unity

  • 978 views
Uploaded on

Unity3D is a great authoring tool to create 3D applications. …

Unity3D is a great authoring tool to create 3D applications.
Regarding immersive virtual reality, its capabilities are limited.
We will discuss the challenges and solutions to create great VR applications in Unity!

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
978
On Slideshare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
167
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. The Challenges of! Creating VR Applications! in Unity Sébastien Kuntz! Sebastien.Kuntz@imin-vr.com! CEO
  • 2. Sébastien Kuntz ! • Founder & President! • 12 years in VR! ! • - French railways! ! •
  • 3. • Simplify VR! • Creation! • Deployment! • So YOU can spend more time creating applications
  • 4. Plan • Unity! • Challenges! • Presence! • Device management! • Display management! • Cluster management! • Interactions! • Deployment! • Rendering speed! • MiddleVR for Unity! • Conclusion
  • 5. Plan Unity! • Challenges! • Presence! • Device management! • Display management! • Cluster management! • Interactions! • Deployment! • Speed! • MiddleVR for Unity! • Conclusion
  • 6. • Platform for creating interactive 3d applications! • Widely used! • 2 million registered developers! • Certified by the US Army and Air Force! • Clients : Bigpoint, Cartoon Network, CocaCola, Disney, Electronic Arts, LEGO, Microsoft, NASA, Nickelodeon, Ubisoft, US Army, Warner Bros.
  • 7. • High-quality rendering! • Deferred renderer! • Occlusion culling! • Realtime shadows! • Ambient occlusion! • Light mapping
  • 8. Unity 3D Features • Programming! • C#, Javascript! 3D formats ! • Profiler! • Physics! • NVidia PhysX! • Cloth! • Soft/rigid bodies! • Ragdolls! • Cars FBX, Collada, 3DS, OBJ…! Audio engine! Networking! Terrain
  • 9. Editor • Simple to use! • Create your own tools inside Unity
  • 10. Asset store • Buy / sell assets! • 3D models! • Textures! • Shaders! • Scripts! • Editor plugins
  • 11. Desktop
  • 12. Plan • Unity! Challenges! • Presence! • Device management! • Display management! • Cluster management! • Interactions! • Deployment! • Speed! • MiddleVR for Unity! • Conclusion
  • 13. Plan • Unity! • Challenges! Presence! • Device management! • Display management! • Cluster management! • Interactions! • Deployment! • Speed! • MiddleVR for Unity! • Conclusion
  • 14. Virtual reality Presence in a virtual world (c) eMagin (c) Barco
  • 15. Experience of presence is in your brain !
  • 16. Cognitive presence Perceptive presence
  • 17. To create and maintain presence, you need to ... • Build hardware • Build software • Build user experience
  • 18. Plan • Unity! • Challenges! • Presence! Device management! • Display management! • Cluster management! • Interactions! • Deployment! • Speed! • MiddleVR for Unity! • Conclusion
  • 19. Device management Physical device! ! Driver Application
  • 20. Device management Tracker 1! Driver Tracker 1 Application Tracker 2! ! Driver Tracker 2
  • 21. Plan • Unity! • Challenges! • Presence! • Device management! Display management! • Cluster management! • Interactions! • Deployment! • Speed! • MiddleVR for Unity! • Conclusion
  • 22. Correct perspective (c) Johnny Lee
  • 23. Correct perspective
  • 24. Stereoscopy
  • 25. Warping & Blending
  • 26. Graphics cards limitations • Number of outputs! • Processing power
  • 27. multiple graphics cards • Pro! • More outputs! • More processing power! • Con! • 3D engines are not optimized for multi-pipe
  • 28. Cluster • Pro! • More outputs! • More processing power! • Con! • 3D engines don’t support clustering
  • 29. Cluster • No seams across displays require 3 layers of sync! • 3D objects synchro (Scenelock)! • New images display (Swaplock)! • Left/Right eyes (Genlock)
  • 30. Plan • Unity! • Challenges! • Presence! • Device management! • Display management! • Cluster management! Interactions! • Deployment! • Speed! • MiddleVR for Unity! • Conclusion
  • 31. Interactions • Many different 3D user interactions! • Navigation! • Selection! • Manipulation! • Draw! • System control! • etc.
  • 32. Interactions are dependent on hardware
  • 33. Menus
  • 34. Haptics (force feedback)
  • 35. Plan • Unity! • Challenges! • Presence! • Device management! • Display management! • Cluster management! • Interactions! Deployment! • Speed! • MiddleVR for Unity! • Conclusion
  • 36. Deployment • Modify application to! • Change drivers! • Change displays! • Change interactions
  • 37. Plan • Unity! • Challenges! • Presence! • Device management! • Display management! • Cluster management! • Interactions! • Deployment! Rendering speed! • MiddleVR for Unity! • Conclusion
  • 38. Rendering speed
  • 39. Plan • Unity! Challenges! • Presence! • Device management! • Display management! • Cluster management! • Interactions! • Deployment! • Rendering Speed! • MiddleVR for Unity! • Conclusion
  • 40. Application
  • 41. Plan • Unity! • Challenges! • Presence! • Device management! • Display management! • Cluster management! • Interactions! • Deployment! • Speed! MiddleVR for Unity! • Conclusion
  • 42. To create and maintain presence, you, the author, need to ... • Build hardware • Build software • Build user experience
  • 43. “VR is mainly a software problem”! Jaron Lanier
  • 44. What is MiddleVR ? • VR plugin ! • Functionalities ○Manage input devices •3D trackers, keyboard, mouse, joystick ! ○Manage display •Viewports, cameras, stereoscopy, cluster
  • 45. Desktop
  • 46. MiddleVR tutorial http://www.youtube.com/watch?v=EEKH-1EilXs
  • 47. Device management
  • 48. Device management
  • 49. Supported Devices • Via VRPN : • A.R.T • Vicon • Optitrack • Wiimote ! • Polhemus Fastrak Tracker, 3Space Tracker, Patriot Trackers, • • • • • • • • Ascension Flock of Bird Intersense IS600, IS900 3dTech HiBall-3000 Wide Area Tracker, Advanced Realtime Tracking Gmbh DTrack Client, World Viz Precision Position Tracker PPT 1.2, Natural Point Optitrack Rigid Body Toolkit, Phasespace optical tracking OWL, ... • Native drivers • Oculus Rift • Leap Motion • Kinect (Microsoft SDK) • PNI SpacePoint Fusion • GameTrak • Razer Hydra • Motion Analysis (beta)
  • 50. First integration of Haption’s haptic devices http://www.youtube.com/watch?v=lnoE_VclnOI
  • 51. Stereoscopy • Active stereoscopy (OpenGL Quad-Buffer)! • Which Unity can’t natively handle! • Passive stereoscopy
  • 52. Cluster & Multi-pipe • Scenelock! • Swaplock! • Genlock! • Multi-pipe (soon)
  • 53. Supported displays • • • • • • • • • Virtually any HMD VR-Wall, Powerwall Workbench, Holobench HoloStage CAVEs 3D TVs zSpace Igloo Vision …
  • 54. Deployment • Abstractions! • Devices! • 3D Nodes (User)! • Interactions (soon)
  • 55. Device management Driver Tracker 1 Application Driver Tracker 2
  • 56. Device abstraction Driver Tracker 1 Virtual Device Driver Tracker 2 Application
  • 57. Device management Driver Replay Simulation Virtual Device Application
  • 58. User abstraction Virtual Device 1 Virtual Device 2 Inverse Kinematics Hand (3D Node) Application
  • 59. Interactions • MiddleVR includes high-level interactions! • Navigation! • Selection! • Manipulation! • … More to come! • Future: Interactions abstractions! • Dynamically change interactions based on hardware
  • 60. Interactions abstraction? Grab-air Steering Navigation Point Raycasting Hand Selection Go-go Homer WIM Hand Manipulation Application
  • 61. Menus • Ability to create menus and GUIs in HTML5 (soon)
  • 62. • Unity! Speed • Many options for optimizations! • Occlusion culling! • Static batching! • Lightmap baking! • etc.! • 32-bit editor only! • But player is 32-bit & 64-bit !! • Subdivide parts in editor! • Compositing! • Cloud
  • 63. Compositing Stereoscopy
  • 64. Compositing Image decomposition
  • 65. VR in the Cloud • Temporary increase in power! • Combine with compositing
  • 66. Perceptive illusions • Playing with limitations of human perception! • Sensitivity of senses! • Selectivity of attention! • Useful to lower cost/complexity of VR systems
  • 67. Perceptive illusions Redirected walking (c) Franck Steinicke - Wurzburg University
  • 68. Perceptive illusions Change blindness Change blindness : (c) Evan Suma - USC/ICT
  • 69. Conclusion ! ! Thank you ! Questions ? ! Sebastien.Kuntz@imin-vr.com