Immersive Virtual Reality

Immersive Virtual Reality Sébastien Kuntz A VR Geek’s blog http://cb.nowan.net/blog Last Update : 13 Dec 2010Monday, December 13, 2010

Licence This “Immersive VR introduction” by Sebastien Kuntz is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/ You are free: to Share — to copy, distribute and transmit the work to Remix — to adapt the work Under the following conditions: Attribution — You must attribute “Immersive VR course” to (with link): “ Sebastien Kuntz (A VR Geek’s blog - http://cb.nowan.net/blog) “Monday, December 13, 2010

Objectives Give you a tour of the IVR world Deﬁnitions Systems Technique (low-level) Hardware Applications Technique (high-level) Ecosystem ChallengesMonday, December 13, 2010

What is VR for you ?Monday, December 13, 2010

Who am I ? Software engineer 4 years at French Railways (SNCF) VR Engine Fiacre (c) SNCF Two simulators A VR Geek’s blog http://cb.nowan.net/blog Twitter : @VRGeek Simurat (c) SNCFMonday, December 13, 2010

4 years Virtools / Dassault Systèmes VR Lead Engineer Virtools / 3dvia Studio Co-organizer of the ﬁrst AFRV : Board member http://www.afrv.fr VR Geeks association http://www.vrgeeks.orgMonday, December 13, 2010

Why now ?Monday, December 13, 2010

Why now ? Already changing world in industry and researchMonday, December 13, 2010

Why now ? Already changing world in industry and research DemocratizationMonday, December 13, 2010

Why now ? Already changing world in industry and research Democratization Are we in the future ? No ! We’re still in prehistory !Monday, December 13, 2010

Why now ? Already changing world in industry and research Democratization Are we in the future ? No ! We’re still in prehistory ! WE can make a difference, invent systems, applications, businesses, interactionsMonday, December 13, 2010

Books, movies, games ? VR ?Monday, December 13, 2010

What is (not) VR ?Monday, December 13, 2010

What is (not) VR ? VR = iVR + Metaverse (Gibson, Stephenson)Monday, December 13, 2010

What is (not) VR ? VR = iVR + Metaverse (Gibson, Stephenson) Metaverse = SecondLife, World of WarcraftMonday, December 13, 2010

What is (not) VR ? VR = iVR + Metaverse (Gibson, Stephenson) Metaverse = SecondLife, World of Warcraft Natural Reality vs VRMonday, December 13, 2010

Immersive VR = iVR iVR is the science and technology required for a user to feel present, via perceptive, cognitive and functional immersion and interaction, in a (computer) generated environment.Monday, December 13, 2010

What is VR ? Realtime 3D != VR Stereoscopy != VR 3D Tracking != VR But everything Haptics != VR combined is VR 3D Sound != VR 3D Interactions != VRMonday, December 13, 2010

Presence “Perceptual illusion of non mediation” Participant forget that their perception is mediated by technology Mel Slater Place illusion = Perceptive Immersion Plausibility illusion = Cognitive ImmersionMonday, December 13, 2010

Place Illusion Sense of “being there” Perceptive immersion When Place Illusion (PI) breaks, it can quickly recover A function of how the perceptual system normally works Action <> Perception loopMonday, December 13, 2010

Plausibility Illusion The illusion that what is apparently happening is really happening Cognitive immersion Your actions have effects on the VE, other events of the VE affect your sensations, and that these events are credible When Plausibility Illusion (Psi) breaks, it is unlikely to recover. You have ceased to accept the ‘reality’Monday, December 13, 2010

Once you get and maintain both Place Illusion and Plausibility Illusion, you are completely involved emotionally, physically and psychologically. Natural reactions Fear, blush, smile at an avatarMonday, December 13, 2010

Books, movies, games ? “[Those] require deliberate attention, and “presence” is not automatic – it is not simply a function of how the perceptual system normally works, but is something that essentially needs to be learned (…) PI may still be reported, but this is as a consequence of additional creative mental processing.” - Mel Slater ”Having a cognitive activity in a virtual environment without a physical activity (sensori-motor)is outside the boundaries of VR” - Le traité de la RVMonday, December 13, 2010

Measure presence ?Monday, December 13, 2010

Measure presence ? Can’t objectively measure presence (qualia) What you can measure is if people respond to this Virtual Environment (VE) as if it is real, which is a result of presence.Monday, December 13, 2010

Immersion Immersion is a technical capability of a VR system A user will experience PI if the system has a good level of immersion. “a property of the valid actions that are possible within the system. Generally, system A is at a higher level of immersion than system B if the valid actions of B form a proper subset of those of A.”Monday, December 13, 2010

Immersion parameters Graphics framerate Extent of tracking Tracking latency (measure?) Quality of the images Field of view Plausibility of behaviors Range of sensory modalities ...Monday, December 13, 2010

Ridiculous If it’s real VR, you probably look ridiculousMonday, December 13, 2010

Paul MilgramMonday, December 13, 2010

Summary Immersive Virtual Reality = Presence Presence Perceptive (body) + Cognitive (mind) immersion Results in natural reactionsMonday, December 13, 2010

VR SystemsMonday, December 13, 2010

Cave http://www.youtube.com/watch?v=pagVgeFaR5Q Clarte - AtriumMonday, December 13, 2010

Monday, December 13, 2010

http://www.youtube.com/watch?v=Hb29bLHEjUg Clarte - SAS CubeMonday, December 13, 2010

Cave (c) Sébastien KuntzMonday, December 13, 2010

LIVES Lifelike Immersive Virtual Environment Space Dassault Systèmes 3 sides 7 HD Barco ProjectorsMonday, December 13, 2010

Calit2 - StarCaveMonday, December 13, 2010

Projector based (c) Arnaud MutheletMonday, December 13, 2010

Projector based Workbench © PSA Peugeot Citroën Mediatheque.Monday, December 13, 2010

Projector basedMonday, December 13, 2010

Panoscope 360°Monday, December 13, 2010

La GéodeMonday, December 13, 2010

jDome http://www.youtube.com/watch?v=rNsFSVhZi9wMonday, December 13, 2010

Clarte R-Screen http://www.youtube.com/watch?v=AcV5ma5nEVgMonday, December 13, 2010

Screen based Full-scale NexCAVE developed and deployed by Calit2 for Saudi Arabias King Abdullah University of Science and Technology - http://www.calit2.net/newsroom/release.php?id=1743Monday, December 13, 2010

Projector based VR Systems Pros +/- Collaborative Not much to wear High resolution, big FOV Cons (Very) expensive ! Lots of space ! ( front and behind, AC... ) Not 360° Cluster hard to program !Monday, December 13, 2010

VRKit-Wall 1000€Monday, December 13, 2010

VR Kit - Wall http://www.youtube.com/watch?v=oDlCEOlBhp0&Monday, December 13, 2010

VRKit-Wall - 1000€ http://cb.nowan.net/blog/2010/07/02/vr-kit-wall-a-dream-come-true/Monday, December 13, 2010

Limitations Shadows Cables StereoscopyMonday, December 13, 2010

Improvements Better Resolution Wireless tracking Ground screenMonday, December 13, 2010

HMD Head mounted displaysMonday, December 13, 2010

VRKit-HMD 600€ Limitations Cables Resolution/FOV Improvements Wireless Everything in backpack http://cb.nowan.net/blog/2010/06/30/vr-kit-hmd-portable-vr/Monday, December 13, 2010

VRKit - HMD http://www.youtube.com/watch?v=MSp8V9pu_wAMonday, December 13, 2010

HMD Pros Lightweight Compact Easier to program 360° tracking Cons Low resolution, low FOV Must wear, adjustMonday, December 13, 2010

Augmented Reality Systems Reality + 3DMonday, December 13, 2010

Summary Projector/screen based Cave, holostage, workbench Collaborative, high FOV Expensive, Complex HMD 360° “Cheap” Low Fov/resolutionMonday, December 13, 2010

Stereoscopy Do you really need stereoscopy ?Monday, December 13, 2010

Stereoscopy ? http://www.youtube.com/watch?v=Jd3-eiid-Uw Johnny Lee - Head Tracking with a wiimoteMonday, December 13, 2010

Stereoscopy ? http://www.youtube.com/watch?v=nRYybo2WNwU Esiea Virtual ShowcaseMonday, December 13, 2010

Stereoscopy You perceived depth without stereoscopy !Monday, December 13, 2010

Depth perception Monoscopic cues : Stereoscopic cues : perspective eye accomodation atmospheric artifacts (fog) relative sizes eye convergence angle occlusion parallax disparity between left shadows and right images Parallax can be more important for depth perception! Stereoscopy is important for size and distance evaluationMonday, December 13, 2010

Creating stereoscopic images Render left and right images manually OpenGL QuadBuffer Theoretically impossible in DirectXMonday, December 13, 2010

Creating stereoscopic images Driver computes left/right pair automatically NVidia 3D Vision IZ3D, DDDMonday, December 13, 2010

3D Vision Issues GUI Mouse Cursor View dependent shaders Incorrect stereo for true VR Can’t turn head Static convergence distanceMonday, December 13, 2010

Creating stereoscopic images Software “Hacks” : Philips WoW, TriovizMonday, December 13, 2010

3D picture reconstructed from frame buffer + depth buffer http://www.youtube.com/watch?v=7QrnwoO1-8A& Oliver Kreylos - 3D Video with KinectMonday, December 13, 2010

Different stereoscopic rendering modes Active Passive Image composition Checkerboard Line interleaved Side/Side, Top/Bottom AnaglypheMonday, December 13, 2010

Different stereoscopic separation schemes Active shutter glasses Polarized glasses Anaglyph Inﬁtec Autostereoscopic HologramsMonday, December 13, 2010

Active stereoscopyMonday, December 13, 2010

Passive Stereoscopy Linear Circular Polarization PolarizationMonday, December 13, 2010

Anaglyph http://upload.wikimedia.org/wikipedia/commons/0/0c/Stereograph_as_an_educator_-_anaglyph.jpgMonday, December 13, 2010

Inﬁtec - Eye separation by color spectrum -Monday, December 13, 2010

Light Efﬁciency http://www.barco.com/projection_systems/downloads/ Stereoscopic_technologies_overview.pdfMonday, December 13, 2010

AutostereoscopicMonday, December 13, 2010

Holograms http://www.youtube.com/watch?v=Lc3EN56JxR8Monday, December 13, 2010

Holograms http://www.youtube.com/watch?v=VVFoJxdJKBQMonday, December 13, 2010

Holograms Stores ALL light interactions through an entire “window”Monday, December 13, 2010

Holograms - look at the magniﬁying glass http://www.youtube.com/watch?v=F7or4j4RWCwMonday, December 13, 2010

Stereoscopy for multiple users ?Monday, December 13, 2010

Stereoscopy for multiple users ? Active + polarized SequentialMonday, December 13, 2010

Multi-displayMonday, December 13, 2010

What can we do with one graphics card?Monday, December 13, 2010

Nvidia SLI - ATI Crossﬁre Use two graphics card for output on only one graphics cards Alternate frame rendering Split frame renderingMonday, December 13, 2010

Scalable graphics http://www.ScalableGraphics.com Image decomposition ( ~= SLI ) Useful if application is fill-rate limitedMonday, December 13, 2010

Scalable graphics Database decomposition Useful if application is CPU limited frame rate decreases with increasing size of 3D scenesMonday, December 13, 2010

Scalable graphics StereoscopyMonday, December 13, 2010

One particular Scalable Graphics Setup Linux Stereo Inﬁniband Inﬁniband Windows Windows Left eye Right eyeMonday, December 13, 2010

How many displays with one computer ?Monday, December 13, 2010

Why multi-display ? Multiple display surfaces Cave Increase display size Increase brightness CPU/GPU Bottlenecks ClusterMonday, December 13, 2010

How ? Matrox DualHead2Go Matrox TripleHead2Go Multiple graphics card in a single PC Often max 2 graphics cards CyvizMonday, December 13, 2010

Multiple graphics cards in one PC Multi-pipe ~= Multi-GPU One rendering context per GPU Problems of resources sharing, threading Need speciﬁc graphics engine adaptations But best performancesMonday, December 13, 2010

NVidia Quadro Plex•Up to : •4 GPU •4 outputs •16gb memory•Can combine two Quadro PlexMonday, December 13, 2010

Quadro Mosaic Mode Mosaic mode hides multi-gpu Only shows one gpu to system Can have 4 outputs with 1 plex without needing multi-pipe adaptation Good performancesMonday, December 13, 2010

ATI EyeFinityMonday, December 13, 2010

Increase size or brightness Mersive autocalibration http://www.dailymotion.com/video/x3bpxu_mersives-amazing-displayMonday, December 13, 2010

What if need more displays ? 96 projectors ?Monday, December 13, 2010

Cluster !Monday, December 13, 2010

Cluster Pros Scalable Each node can be cheap Easy node replacement Cons Complex to manage/programMonday, December 13, 2010

Cluster Synchronisation Scene synchronisation (Scenelock/Framelock) Swap-Buffer synchronisation (Swaplock) Eyes synchronisation (Genlock)Monday, December 13, 2010

Cluster Scene synchronisation Scenelock == Framelock Make sure the scene is coherent on all computers Automatic clustering TechViz OpenGL driver interceptor Sends all OpenGL commands to all computers Semi-Automatic clustering Virtools Tag all objects and variables that need synchro Cause / Consequences ?Monday, December 13, 2010

Cluster Swap-buffer synchronisation Swaplock Hardware Old graphics cards (Wildcats) Quadros with G-Sync add-on card SoftwareMonday, December 13, 2010

Left/Right eye synchro Genlock Crucial for stereoscopy Can only be done in hardware Only on pro cards (Quadro, FireGL) Quadro G-Sync card Signal also sent to glasses by infraredMonday, December 13, 2010

Stereo glasses / Optical tracking See any problem ?Monday, December 13, 2010

Tracking Mechanical Fakespace’s Boom Pros Precise Fast Help Lift weight Cons Big, discontinuedMonday, December 13, 2010

Tracking Magnetic Polhemus Ascension Pros Precise, fast Cons Wires Sensitive to metalMonday, December 13, 2010

Tracking Inertial SpacePoint Fusion XSens Mocap http://www.youtube.com/watch?v=x8xvUvgd1zQ SpacePoint Fusion Demo Accelerometers, Gyroscopes, magnetometers Pros Very fast Light Cons Drift Orientation onlyMonday, December 13, 2010

Tracking Optical ART ioTracker OptiTrack TrackIR Pros Wireless No electronics Cons Install cameras Line of sightMonday, December 13, 2010

Tracking Mixing Optical Intersense IS-900 Inertial + Ultrasound IS-1200 Inertial + AR Patterns Playstation Move Inertial + OpticalMonday, December 13, 2010

Optical Body Tracking 4D Views http://www.4dviews.com http://www.youtube.com/watch?v=ZG53YDVW1IkMonday, December 13, 2010

http://www.youtube.com/watch?v=yHo4QPBl1H0 http://www.youtube.com/watch?v=_UAAhvFMdekMonday, December 13, 2010

GlovesMonday, December 13, 2010

Low-cost hardware Gametrak TrackIR Wiimote AR tracking Playstation Move Microsoft KinectMonday, December 13, 2010

http://www.youtube.com/watch?v=s9ybHddDMgM Playstation MoveMonday, December 13, 2010

HapticsForce feedback http://www.youtube.com/watch?v=WJ1Rz2clLS0 Novint FalconMonday, December 13, 2010

Haptics Force feedback http://www.youtube.com/watch?v=Hb29bLHEjUg SAS Cube & Spidar (starts at 1min 36s)Monday, December 13, 2010

Haptics Force feedbackMonday, December 13, 2010

TN Games - Gaming VestMonday, December 13, 2010

Locomotion Virtusphere http://www.youtube.com/watch?v=qTnnJR-hS7k CirculaFloor http://www.youtube.com/watch?v=rYsvB2y2Ero OmniDirectionnal Treadmill http://www.youtube.com/watch?v=BQw1tsgrJOsMonday, December 13, 2010

Using Devices Direct access to drivers Devices abstraction User abstractionMonday, December 13, 2010

Using Devices Direct access to drivers Pros Access to all driver functionnalities Minimum overhead Cons Driver Application Little portabilityMonday, December 13, 2010

Devices abstraction Multiple drivers offer similar interface Tracker::GetPosition Provide a common interface to different devices Pros Portability Replay Driver Tracker Application Emulation Cons Overhead (?) Lowest common denominatorMonday, December 13, 2010

VRPN OpenSource VRPN Device abstraction Driver Server Tracker Network Analog Button VRPN Application Tracker Client/ServerMonday, December 13, 2010

OpenTrackerMonday, December 13, 2010

Body abstraction Tracker 0 Head Application Tracker 1 HandMonday, December 13, 2010

IVR Applications Design, prototyping Marketing Training Medical Research Geoscience, ﬁnance, archeologie, urbanism, sport, art...Monday, December 13, 2010

iVR Applications Design, prototyping Car, aircraft, boat, tractors, house appliances Collaborative design review Experience from user’s point of view (c) Peugeot PSA Easy and fast to prototype More iterations, better products Renault, Peugeot, Ford, Mercedes, BMW, John Deere, DCNS, Airbus, Boeing ... (c) MieleMonday, December 13, 2010

http://www.youtube.com/watch?v=XECqdtdY-Gc Clarte - VR4DMonday, December 13, 2010

FORDMonday, December 13, 2010

JaguarMonday, December 13, 2010

iVR Applications Marketing Product experience (c) University of Gronigen Shopping experience Procter&Gamble Kimberley Clarke (c) VirtoolsMonday, December 13, 2010

iVR Applications Training Immersion decreases distance from training to reality Same gestures in VR and reality Controlled environment Repeatability, replay Can simplify interaction (c) SNCF - Fiacre Can show invisible phenomenons (mechanics, gaz...) Can help gesture (haptics) Can practice dangerous situations Social interactionsMonday, December 13, 2010

SNCF-SimuratMonday, December 13, 2010

http://www.youtube.com/watch?v=FC0gHv-Snmw ESCIN - Sivic - Crime scene investigationMonday, December 13, 2010

iVR Applications Research Understand Perception, cognition, action Low level (Self-perception, Depth perception, Movement perception, …) High level Self-perception Modiﬁed by avatars (body and self- conﬁdence…) Body Swap Social perception Detect schizophrenia Controlled environment Dissociating stimuli (Balance)Monday, December 13, 2010

iVR Applications Research Heal Exposure therapy PTSD, Phobias Rehabilitation : motor, addictions Create Empathy http://www.youtube.com/watch?v=SinlgK5Q4v4 Pain Distraction Crescent - AcrophobiaMonday, December 13, 2010

Summary IVR strengths Controlled environment Natural reactions Visualize & modify complex data in real-timeMonday, December 13, 2010

AR Applications (c) http://www.igd.fraunhofer.de/igd-a2/ http://www.youtube.com/watch?v=P9KPJlA5yds improve/ BMW Augmented Reality visionMonday, December 13, 2010

iVR Games ? Cave Quake 3 (c) Paul Rajlich Cave Unreal Tournament (c) Jeffrey JacobsonMonday, December 13, 2010

http://www.youtube.com/watch?v=Cqh_F4iN8pg Quake 3 in a panoramic displayMonday, December 13, 2010

http://www.youtube.com/watch?v=MrbwiwGTj0U Clarte’s Atrium in a HMDMonday, December 13, 2010

http://www.youtube.com/watch?v=EMNoaiFX4ZA Immersive Rail Shooter - David ArenouMonday, December 13, 2010

AR Games •http://www.youtube.com/watch?v=EMNoaiFX4ZA •ARrrrrhhhMonday, December 13, 2010

•http://www.youtube.com/watch?v=TzCMAgVrPFc •Dassault Systèmes ChocapicMonday, December 13, 2010

•http://www.youtube.com/watch?v=0w1rAjLnoGM •Frantz Lasorne’s ScopeMonday, December 13, 2010

VR Techniques 3D User Interactions Bowman classiﬁcation for generic interactions Selection Specifying one or more objects from a set Manipulation Modifying object properties (position, orientation, scale, shape, color, texture, behavior, etc.) Navigation Travel Wayﬁnding System Control Menus Issuing command Changing mode Choosing tool Symbolic Input Other interactions Measuring …? http://www.3dui.orgMonday, December 13, 2010

Navidget http://www.youtube.com/watch?v=wpaKjLZll80 NavidgetMonday, December 13, 2010

http://cb.nowan.net/blog/2010/03/25/3dui-grand-prize-2010-the-winners/Monday, December 13, 2010

http://www.youtube.com/watch?v=K90DOeaKMBs VirginiaTechMonday, December 13, 2010

http://www.youtube.com/watch?v=2U_I6mKjWf8 im.veMonday, December 13, 2010

2011 - Solve the puzzleMonday, December 13, 2010

http://www.youtube.com/watch?v=mwwjnCjrTMk VRM - DivaMonday, December 13, 2010

Perceptive Illusions Play with perception’s limits Our perception is dominated by visual sense Balance Redirected Walking : Frank Steinicke - “Analyses of Human Sensitity to Redirected Walking” http://cb.nowan.net/blog/2010/03/29/ieee-vr-2010-perceptive-illusions/Monday, December 13, 2010

Perceptive illusions Change blindness : Evan A. Suma -Exploiting Change Blindness to Expand Walkable Space in a Virtual Environment - IEEE VR 2010Monday, December 13, 2010

Perceptive Ilusions Pseudo haptics http://www.irisa.fr/tactilesMonday, December 13, 2010

Perceptive Illusions Passive haptics Luv Kohli for “Redirected Touching: Warping Space to Remap Passive Haptics”Monday, December 13, 2010

Passive haptics Ford PVM (c) FordMonday, December 13, 2010

Perceptive illusions Learn about human perception through its limitations Improve VR experience by taking advantage of those limitationsMonday, December 13, 2010

http://www.Afrv.frMonday, December 13, 2010

FR - http://www.youtube.com/watch?v=J8wl_EBtLj4 EN - http://www.youtube.com/watch?v=lHZltjDZBJo AFRVMonday, December 13, 2010

VR Geeks http://www.vrgeeks.orgMonday, December 13, 2010

VR Challenges What we must solve VR is mainly a SOFTWARE problem “3DUI is a hard and unexplored software engineering (and ergonomic) problem” Markerless tracking Minimum latency HMDs Lightweight Wide FOV Wireless Full body haptics Multi-sensorial experience Locomotion Avatars VR For AllMonday, December 13, 2010

VR “Experiences” Your goal : Create presence And don’t break it !!! http://www.youtube.com/watch?v=ZBfYIefenp0 ESCIN - Verdun 1916Monday, December 13, 2010

Jaron Lanier’s message to VR Geeks http://www.youtube.com/watch?v=DPYEZAHqotwMonday, December 13, 2010

World Builder http://www.youtube.com/watch?v=VzFpg271sm8Monday, December 13, 2010

Thank you ! Questions ? sebastien.kuntz@nowan.netMonday, December 13, 2010

http://cb.nowan.net	476
http://www.netvibes.com	4
http://static.slidesharecdn.com	3
http://twitter.com	2

Immersive Virtual Reality

by SebKuntz on Nov 23, 2010

Accessibility

Tags

Upload Details

Usage Rights

4 Embeds 485

Statistics

Immersive Virtual Reality — Presentation Transcript