0
Sébastien „Cb‟ Kuntz
@Cb_VRGeek
Philosophy and architecture
SEARIS 2014
Plan
• Introduction
• Origin of MiddleVR
• Goals
• Demo
• Philosophy & Architecture
• Conclusion
(French railways)
“Creating Virtual Reality Games:
The Fundamentals”
Founder & President
Sébastien „Cb‟ Kuntz
Zone hunter (Virtuality-1994)
FRENCH RAILWAYS - SNCF
• Polhemus magnetic tracker, treadmill, 5DT Glove
• C++ / OpenSceneGraph / VRPN / OpenAL
FRENCH RAILWAYS - SNCF
VIRTOOLS
VIRTOOLS VR PACK
• VR tools to simplify VR
Virtual reality
Presence in a virtual world
Ridiculous
If it‟s real
VR, you
probably look
ridiculous !
• Build hardware
To create and maintain presence,
you need to ...
• Build software
• Build user experience
The challenges of creating a VR software
“VR is mainly a software problem”
Jaron Lanier
Device management
Physical device Driver Application
Device management
Tracker 1
Driver Tracker 1
Application
Tracker 2 Driver Tracker 2
Correct perspective
(c) Johnny Lee
Correct perspective
Stereoscopy
Warping & Blending
Warping & Blending
Graphics cards limitations
• Number of outputs
• Processing power
multiple graphics
cards
• Pro
• More outputs
• More processing power
• Con
• 3D engines are not optimized for multi-pipe
Cluster
• Pro
• More outputs
• More processing power
• Con
• 3D engines don‟t support clustering
• No seams across displays require 3 layers of sync
• 3D objects synchro (Scenelock)
• New images display (Swaplock)
• Lef...
Interactions
• Many different 3D user interactions
• Navigation
• Selection
• Manipulation
• Draw
• System control
• etc.
Interactions are dependent
on hardware
Menus
Haptics
(force feedback)
Deployment
• Modify application to
• Change drivers
• Change displays
• Change interactions
• Platform for creating
interactive 3d applications
• Widely used
• 2.5 million registered
developers
• High-quality rendering
• Deferred renderer
• Occlusion culling
• Realtime shadows
• Ambient occlusion
• Light mapping
Unity 3D
Features
• Programming
• C#, Javascript
• Profiler
• Physics
• NVidia PhysX
• Cloth
• Soft/rigid bodies
• Ragdoll...
Editor
• Simple to use
• Create your own tools inside Unity
Asset store
• Buy / sell assets
• 3D models
• Textures
• Shaders
• Scripts
• Editor plugins
Desktop
• Build hardware
To create and maintain presence,
you, the author, need to ...
• Build software
• Build user experience
Existing VR software
• VRPN
• OpenTracker
• VRJuggler
• Virtools
• Worldviz
• …
VRPN
VR Juggler
VIRTOOLS VR PACK
Worldviz Vizard
• Based on OpenSceneGraph
• Python
Worldviz Vizard
The need for better
VR software tools
• Hardware getting cheaper
• But engineers are not getting cheaper
• Need to be more...
Goals
• Simplify VR
• Simplify creation of VR applications
• Simplify deployment of VR applications
• Create new VR tools
...
What is ?
• Generic VR plugin (C++ SDK)
• Functionalities
○Manage input devices
•3D trackers, keyboard, mouse, joystick
○M...
Architecture
MiddleVR (C++)
Drivers OpenGL DirectX Network
C# Wrapper
GUI Unity
Lua
Wrapper
Python
Wrapper
MiddleVR Features
Supported Devices
• Via VRPN :
• A.R.T
• Vicon
• Optitrack
• Wiimote
• Polhemus Fastrak Tracker, 3Space
Tracker, Patriot T...
Haptics
(force feedback)
VR Camera
• MiddleVR computes all the cameras‟ parameters
automatically
• Correct perspective
• Scale 1
Stereoscopy
• Active stereoscopy
(OpenGL Quad-Buffer)
• Which Unity can‟t
natively handle
• Passive stereoscopy
Viewports management
Simply configure your system
Navigations
Homography
Cluster & Multi-pipe
• Scenelock
• Swaplock
• Genlock
• Multi-pipe (soon)
Supported displays
• Virtually any HMD
• VR-Wall, Powerwall
• Workbench, Holobench
• HoloStage
• CAVEs
• 3D TVs
• zSpace
•...
Goals
• Simplify creation of VR applications
• Simplicity of adding VR into application
• Simplicity of API
• High-level t...
Example of device
management
Driver 1
Driver 2
Device management
MiddleVR
Class reference
The value of Simplicity
The value of simplicity
“Perfection is achieved, not when there is
nothing more to add, but when there is
nothing left to ...
The value of simplicity
• For each feature you add
• Test
• Document
• Support / maintain
• Each feature costs !!
The value of simplicity
• Huge value in simplicity !
• User side
• Faster to learn, less to remember
• Developer side
• Fa...
API in simple english
• Long discussions for naming of functions, classes, variables
• Consistent api
• SetNumButtons, Set...
Simplified API
• Simple but complete API
• Like legos
• Rotate
• Node3D->Rotate( Quat )
• Node3D->Rotate( Euler )
• Node3D...
The value of simplicity
“Simplicity is the ultimate sophistication”
DaVinci
The value of simplicity
• Design user interface for the 80%
• Don‟t ruin the experience for the 20%
• Simple things should...
Optimizations
• Engineers have a strong tendency to over-optimise
• Complexifies code
• Code must be above all readable
• ...
Empathy
“The ability to understand and share the
feelings of another.”
Test with users
Hardware SDK are mostly too complicated
Drivers
• Why not just a few simple functions with
sensible defaults ?
• Init()
• Update()
• GetValue()
• Destroy()
• And ...
Documentation
• User Guide
• Class references
Goals
• Simplify deployment of VR applications
• GUI to configure VR systems
• Dynamic reconfiguration of application
• Sh...
Deployment
• Abstractions
• Devices
• 3D Nodes (User)
• Interactions (soon)
Device management
Driver Tracker 1
Application
Driver Tracker 2
Device abstraction
Virtual
Device
Application
Driver Tracker 1
Driver Tracker 2
Device management
Driver
Virtual
Device
Application
Simulation
Replay
Display
• Cameras & viewport
created dynamically
User
• Human-centric rather than techno-centric
Virtual Device 1
Hand
(3D Node)
Application
Virtual Device 2
Inverse Kinematics
User abstraction
Interactions
• MiddleVR includes high-level interactions
• Navigation
• Selection
• Manipulation
• … More to come
• Future...
Interactions abstraction?
Grab-air
Navigation
Application
Point
Steering
Selection
Manipulation
Raycasting
Go-go
Hand
Home...
• How to create portable interactions ?
• Navigations can be portable
• Except collisions
• Generalise what is generalisab...
Menus
• Ability to create menus and GUIs in HTML5 (soon)
Create immersive HTML5 GUI
C++ Reflexion
• Also for configuration file
• Cluster synchronisation
• Accessing drivers specific information
Working with multiple 3D
engines
• Different coordinate systems
• vrCoordinateSystem( Front, Up, Right )
• +/- X,Y,Z
• Ope...
Working with multiple 3D
engines
• C++
• Compiling
• STL, Boost
• Strings (Wide strings)
• Link issues
• Different compile...
• Simplicity
• Creation
• Deployment
• Adaptability
• Hardware
• Software
Our future
• Games/apps development:
Conclusion
http://www.imin-vr.com/download
Thank you !
Questions ?
Sebastien.Kuntz@imin-vr.com
@Cb_VRGeek
@imin_vr
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
Upcoming SlideShare
Loading in...5
×

IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture

1,232

Published on

MiddleVR is a generic VR plugin.
SEARIS is the "Software engineering and architecture for realtime interactive systems" workshop for IEEEVR.

Published in: Technology, Art & Photos
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,232
On Slideshare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture"

  1. 1. Sébastien „Cb‟ Kuntz @Cb_VRGeek Philosophy and architecture SEARIS 2014
  2. 2. Plan • Introduction • Origin of MiddleVR • Goals • Demo • Philosophy & Architecture • Conclusion
  3. 3. (French railways) “Creating Virtual Reality Games: The Fundamentals” Founder & President Sébastien „Cb‟ Kuntz
  4. 4. Zone hunter (Virtuality-1994)
  5. 5. FRENCH RAILWAYS - SNCF • Polhemus magnetic tracker, treadmill, 5DT Glove • C++ / OpenSceneGraph / VRPN / OpenAL
  6. 6. FRENCH RAILWAYS - SNCF
  7. 7. VIRTOOLS
  8. 8. VIRTOOLS VR PACK
  9. 9. • VR tools to simplify VR
  10. 10. Virtual reality Presence in a virtual world
  11. 11. Ridiculous If it‟s real VR, you probably look ridiculous !
  12. 12. • Build hardware To create and maintain presence, you need to ... • Build software • Build user experience
  13. 13. The challenges of creating a VR software
  14. 14. “VR is mainly a software problem” Jaron Lanier
  15. 15. Device management Physical device Driver Application
  16. 16. Device management Tracker 1 Driver Tracker 1 Application Tracker 2 Driver Tracker 2
  17. 17. Correct perspective (c) Johnny Lee
  18. 18. Correct perspective
  19. 19. Stereoscopy
  20. 20. Warping & Blending
  21. 21. Warping & Blending
  22. 22. Graphics cards limitations • Number of outputs • Processing power
  23. 23. multiple graphics cards • Pro • More outputs • More processing power • Con • 3D engines are not optimized for multi-pipe
  24. 24. Cluster • Pro • More outputs • More processing power • Con • 3D engines don‟t support clustering
  25. 25. • No seams across displays require 3 layers of sync • 3D objects synchro (Scenelock) • New images display (Swaplock) • Left/Right eyes (Genlock) Cluster
  26. 26. Interactions • Many different 3D user interactions • Navigation • Selection • Manipulation • Draw • System control • etc.
  27. 27. Interactions are dependent on hardware
  28. 28. Menus
  29. 29. Haptics (force feedback)
  30. 30. Deployment • Modify application to • Change drivers • Change displays • Change interactions
  31. 31. • Platform for creating interactive 3d applications • Widely used • 2.5 million registered developers
  32. 32. • High-quality rendering • Deferred renderer • Occlusion culling • Realtime shadows • Ambient occlusion • Light mapping
  33. 33. Unity 3D Features • Programming • C#, Javascript • Profiler • Physics • NVidia PhysX • Cloth • Soft/rigid bodies • Ragdolls • Cars 3D formats FBX, Collada, 3DS, OBJ… Audio engine Networking Terrain
  34. 34. Editor • Simple to use • Create your own tools inside Unity
  35. 35. Asset store • Buy / sell assets • 3D models • Textures • Shaders • Scripts • Editor plugins
  36. 36. Desktop
  37. 37. • Build hardware To create and maintain presence, you, the author, need to ... • Build software • Build user experience
  38. 38. Existing VR software • VRPN • OpenTracker • VRJuggler • Virtools • Worldviz • …
  39. 39. VRPN
  40. 40. VR Juggler
  41. 41. VIRTOOLS VR PACK
  42. 42. Worldviz Vizard • Based on OpenSceneGraph • Python
  43. 43. Worldviz Vizard
  44. 44. The need for better VR software tools • Hardware getting cheaper • But engineers are not getting cheaper • Need to be more efficient and spend less time creating apps • Democratization • VR developers are not all engineers anymore
  45. 45. Goals • Simplify VR • Simplify creation of VR applications • Simplify deployment of VR applications • Create new VR tools • Adapting existing 3D applications
  46. 46. What is ? • Generic VR plugin (C++ SDK) • Functionalities ○Manage input devices •3D trackers, keyboard, mouse, joystick ○Manage display •Viewports, cameras, stereoscopy, cluster ○High-level building-blocks ○Interactions ○Menu
  47. 47. Architecture MiddleVR (C++) Drivers OpenGL DirectX Network C# Wrapper GUI Unity Lua Wrapper Python Wrapper
  48. 48. MiddleVR Features
  49. 49. Supported Devices • Via VRPN : • A.R.T • Vicon • Optitrack • Wiimote • Polhemus Fastrak Tracker, 3Space Tracker, Patriot Trackers, • Ascension Flock of Bird • Intersense IS600, IS900 • 3dTech HiBall-3000 Wide Area Tracker, • Advanced Realtime Tracking Gmbh DTrack Client, • World Viz Precision Position Tracker PPT 1.2, • Natural Point Optitrack Rigid Body Toolkit, • Phasespace optical tracking OWL, • ... • Native drivers • Oculus Rift • Leap Motion • Kinect (Microsoft SDK) • PNI SpacePoint Fusion • GameTrak • Razer Hydra • SpaceMouse • Motion Analysis (beta)
  50. 50. Haptics (force feedback)
  51. 51. VR Camera • MiddleVR computes all the cameras‟ parameters automatically • Correct perspective • Scale 1
  52. 52. Stereoscopy • Active stereoscopy (OpenGL Quad-Buffer) • Which Unity can‟t natively handle • Passive stereoscopy
  53. 53. Viewports management
  54. 54. Simply configure your system
  55. 55. Navigations
  56. 56. Homography
  57. 57. Cluster & Multi-pipe • Scenelock • Swaplock • Genlock • Multi-pipe (soon)
  58. 58. Supported displays • Virtually any HMD • VR-Wall, Powerwall • Workbench, Holobench • HoloStage • CAVEs • 3D TVs • zSpace • Igloo Vision • …
  59. 59. Goals • Simplify creation of VR applications • Simplicity of adding VR into application • Simplicity of API • High-level tools • Interactions: Selection, manipulation, navigation, menus, GUIs
  60. 60. Example of device management
  61. 61. Driver 1
  62. 62. Driver 2
  63. 63. Device management
  64. 64. MiddleVR
  65. 65. Class reference
  66. 66. The value of Simplicity
  67. 67. The value of simplicity “Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away” Antoine de St Exupery
  68. 68. The value of simplicity • For each feature you add • Test • Document • Support / maintain • Each feature costs !!
  69. 69. The value of simplicity • Huge value in simplicity ! • User side • Faster to learn, less to remember • Developer side • Faster to code • Easier maintain • Less bugs • Business side • Need less coders • More interesting features • Easy download / setup > Quickly validate on user‟s hardware ! • Easier support
  70. 70. API in simple english • Long discussions for naming of functions, classes, variables • Consistent api • SetNumButtons, SetButtonsNum, SetNbButtons > SetButtonsNb • French not good at english • Analog ? Digital ? Proxy ? • Use natural terms depending on device • Joystick->GetAxisValue() • Joystick->IsButtonPressed() • Keyboard->IsKeyPressed() • Why “Register” ? Why not “Add” ? • Why so complex, want to sound very clever ? • AddDevice, AddCamera, … • API in degrees, because more intuitive / simple to work with
  71. 71. Simplified API • Simple but complete API • Like legos • Rotate • Node3D->Rotate( Quat ) • Node3D->Rotate( Euler ) • Node3D->Rotate( Axis, Angle) • Quat( quat ), Quat( Euler ), Quat( Axis, Angle)
  72. 72. The value of simplicity “Simplicity is the ultimate sophistication” DaVinci
  73. 73. The value of simplicity • Design user interface for the 80% • Don‟t ruin the experience for the 20% • Simple things should be simple • Complex things can be complex • Put in an “advanced menu” • Don‟t let beauty of design ruin user experience • Like real life architects who want their design to be beautiful but totally unusable on a day to day • Example of Lego • Simple but complete
  74. 74. Optimizations • Engineers have a strong tendency to over-optimise • Complexifies code • Code must be above all readable • To avoid bugs • To maintain maintenability • If you want to optimise • First profile your code • Optimize what is *really* taking time • not what you *think* takes time
  75. 75. Empathy “The ability to understand and share the feelings of another.” Test with users
  76. 76. Hardware SDK are mostly too complicated
  77. 77. Drivers • Why not just a few simple functions with sensible defaults ? • Init() • Update() • GetValue() • Destroy() • And add more functions for more advanced
  78. 78. Documentation • User Guide • Class references
  79. 79. Goals • Simplify deployment of VR applications • GUI to configure VR systems • Dynamic reconfiguration of application • Ship with standard configurations • Abstractions
  80. 80. Deployment • Abstractions • Devices • 3D Nodes (User) • Interactions (soon)
  81. 81. Device management Driver Tracker 1 Application Driver Tracker 2
  82. 82. Device abstraction Virtual Device Application Driver Tracker 1 Driver Tracker 2
  83. 83. Device management Driver Virtual Device Application Simulation Replay
  84. 84. Display • Cameras & viewport created dynamically
  85. 85. User • Human-centric rather than techno-centric
  86. 86. Virtual Device 1 Hand (3D Node) Application Virtual Device 2 Inverse Kinematics User abstraction
  87. 87. Interactions • MiddleVR includes high-level interactions • Navigation • Selection • Manipulation • … More to come • Future: Interactions abstractions • Dynamically change interactions based on hardware
  88. 88. Interactions abstraction? Grab-air Navigation Application Point Steering Selection Manipulation Raycasting Go-go Hand Homer Hand WIM
  89. 89. • How to create portable interactions ? • Navigations can be portable • Except collisions • Generalise what is generalisable • + Custom part for engine dependent • Collisions • Highlights • Gizmos Interactions
  90. 90. Menus • Ability to create menus and GUIs in HTML5 (soon)
  91. 91. Create immersive HTML5 GUI
  92. 92. C++ Reflexion • Also for configuration file • Cluster synchronisation • Accessing drivers specific information
  93. 93. Working with multiple 3D engines • Different coordinate systems • vrCoordinateSystem( Front, Up, Right ) • +/- X,Y,Z • OpenGL: • Front= -Z • Up = +Y • Right = +X • DirectX: • Front=+Z • Up=+Y • Right=+X
  94. 94. Working with multiple 3D engines • C++ • Compiling • STL, Boost • Strings (Wide strings) • Link issues • Different compilers (VC2005/2008/2010/2012) • Different languages • C wrapper • Lua wrapper
  95. 95. • Simplicity • Creation • Deployment • Adaptability • Hardware • Software
  96. 96. Our future • Games/apps development:
  97. 97. Conclusion
  98. 98. http://www.imin-vr.com/download Thank you ! Questions ? Sebastien.Kuntz@imin-vr.com @Cb_VRGeek @imin_vr
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×