Uploaded byStanfordComputationalImaging
PPTX, PDF2,763 views

VR2.0: Making Virtual Reality Better Than Reality?

Computational near-eye displays aim to address the vergence-accommodation conflict in virtual reality by dynamically adjusting focus based on a user's gaze. Studies show that adaptive focus displays can improve visual clarity and comfort for most users compared to conventional VR displays. However, presbyopic users over age 40 may still experience some visual issues due to reduced ability to accommodate. Future near-eye displays could explore technologies like multiplane displays and light field displays to better support users of all ages.

Embed presentation

Downloaded 147 times
1 / 114
2 / 114
3 / 114
4 / 114
5 / 114
6 / 114
7 / 114
8 / 114
9 / 114
10 / 114
11 / 114
12 / 114
13 / 114
14 / 114
15 / 114
16 / 114
17 / 114
18 / 114
19 / 114
20 / 114
21 / 114
22 / 114
23 / 114
24 / 114
25 / 114
26 / 114
27 / 114
28 / 114
29 / 114
30 / 114
31 / 114
32 / 114
33 / 114
34 / 114
35 / 114
36 / 114
37 / 114
38 / 114
39 / 114
40 / 114
41 / 114
42 / 114
43 / 114
44 / 114
45 / 114
46 / 114
47 / 114
48 / 114
49 / 114
50 / 114
51 / 114
52 / 114
53 / 114
54 / 114
55 / 114
56 / 114
57 / 114
58 / 114
59 / 114
60 / 114
61 / 114
62 / 114
63 / 114
64 / 114
65 / 114
66 / 114
67 / 114
68 / 114
69 / 114
70 / 114
71 / 114
72 / 114
73 / 114
74 / 114
75 / 114
76 / 114
77 / 114
78 / 114
79 / 114
80 / 114
81 / 114
82 / 114
83 / 114
84 / 114
85 / 114
86 / 114
87 / 114
88 / 114
89 / 114
90 / 114
91 / 114
92 / 114
93 / 114
94 / 114
95 / 114
96 / 114
97 / 114
98 / 114
99 / 114
100 / 114
101 / 114
102 / 114
103 / 114
104 / 114
105 / 114
106 / 114
107 / 114
108 / 114
109 / 114
110 / 114
111 / 114
112 / 114
113 / 114
114 / 114
Making Virtual Reality better than Reality? Gordon Wetzstein Stanford University IS&T Electronic Imaging 2017 www.computationalimaging.org
Personal Computer e.g. Commodore PET 1983 Laptop e.g. Apple MacBook Smartphone e.g. Google Pixel AR/VR e.g. Microsoft Hololens ???
A Brief History of Virtual Reality 1838 1968 2012-2017 Stereoscopes Wheatstone, Brewster, … VR & AR Ivan Sutherland VR explosion Oculus, Sony, HTC, MS, … Nintendo Virtual Boy 1995 VR 2.0
Where we are now IFIXIT teardown
Magnified Display 1 d + 1 d' = 1 f d d’ f
Real World: Vergence & Accommodation Match!
Current VR Displays: Vergence & Accommodation Mismatch for people with normal vision
Presbyopia [Katz et al. 1997] 68% age 80+ 43% age 40 25% Hyperopia [Krachmer et al. 2005] Myopia 41.6% [Vitale et al. 2009] How Many People Have Normal Vision? all numbers of US population
4D / 25cm Optical Infinity Normal vision Nearsighted/myopic Farsighted/Hyperopic Presbyopic Focal range (range of clear vision) Modified from Pamplona et al, Proc. of SIGGRAPH 2010 Nearsightedness & Farsightedness
Computational Near-eye Displays
• Q1: Can computational displays effectively replace glasses in VR/AR? • Q2: How to address the vergence-accommodation conflict for users of different ages? • Q3: What are (in)effective near-eye display technologies? possible solutions: gaze-contingent focus, monovision, multiplane, light field displays, …
• Q1: Can computational displays effectively replace glasses in VR/AR? • Q2: How to address the vergence-accommodation conflict for users of different ages? • Q3: What are (in)effective near-eye display technologies? possible solutions: gaze-contingent focus, monovision, multiplane, light field displays, …
Magnified Display Display Lens Fixed Focus 1 d + 1 d' = 1 f d d’ f
Adaptive Focus Magnified Display Display Lens 1 d + 1 d' = 1 f actuator  vary d’
Adaptive Focus Magnified Display Display Lens focus-tunable lens  vary f 1 d + 1 d' = 1 f
Adaptive Focus - History • M. Heilig “Sensorama”, 1962 (US Patent #3,050,870) • P. Mills, H. Fuchs, S. Pizer “High-Speed Interaction On A Vibrating-Mirror 3D Display”, SPIE 0507 1984 • S. Shiwa, K. Omura, F. Kishino “Proposal for a 3-D display with accommodative compensation: 3DDAC”, JSID 1996 • S. McQuaide, E. Seibel, J. Kelly, B. Schowengerdt, T. Furness “A retinal scanning display system that produces multiple focal planes with a deformable membrane mirror”, Displays 2003 • S. Liu, D. Cheng, H. Hua “An optical see-through head mounted display with addressable focal planes”, Proc. ISMAR 2008 manual focus adjustment Heilig 1962 automatic focus adjustment Mills 1984 deformabe mirrors & lenses McQuaide 2003, Liu 2008
Padmanaban et al., PNAS 2017
Padmanaban et al., PNAS 2017
Padmanaban et al., PNAS 2017
Padmanaban et al., PNAS 2017
Padmanaban et al., PNAS 2017
Padmanaban et al., PNAS 2017
Padmanaban et al., PNAS 2017
at ACM SIGGRAPH 2016 EyeNetra.com
at ACM SIGGRAPH 2016 participants of the study, 152 total EyeNetra.com
Participants - Prescription Padmanaban et al., PNAS 2017 n = 70, ages 21-64
How sharp is the target? (blurry, medium, sharp) Is the target fused? (yes, no) 4D (0.25m) 3D (0.33m) 2D (0.50m) 1D (1m) Four simulated distances Task
far near 1D 1m 2D 0.5m 3D 0.3m 4D 0.25m Distance medium 0 Relativesharpness sharp 1 blurry -1 VR uncorrected VR corrected Results - Sharpness Padmanaban et al., PNAS 2017
far near far near 1D 1m 2D 0.5m 3D 0.3m 4D 0.25m Distance medium 0 sharp 1 blurry -1 Relativesharpness VR uncorrected VR corrected Results - Sharpness Padmanaban et al., PNAS 2017
far near far near 1D 1m 2D 0.5m 3D 0.3m 4D 0.25m Distance medium 0 sharp 1 blurry -1 Relativesharpness VR uncorrected VR corrected Results - Sharpness Padmanaban et al., PNAS 2017
Mean = 0.63 Mean = 0.60 far near far near 1D 1m 2D 0.5m 3D 0.3m 4D 0.25m Distance medium 0 sharp 1 blurry -1 Relativesharpness VR uncorrected VR corrected normal correction Results - Sharpness Padmanaban et al., PNAS 2017
far far near 1D 1m 2D 0.5m 3D 0.3m 4D 0.25m Distance VR uncorrected VR corrected 1 Proportionfused 0.8 0.6 0.4 0.2 0 Results - Fusion Padmanaban et al., PNAS 2017
far far near 1D 1m 2D 0.5m 3D 0.3m 4D 0.25m Distance VR uncorrected VR corrected 1 Proportionfused 0.8 0.6 0.4 0.2 0 Results - Fusion Padmanaban et al., PNAS 2017
Computational Near-eye Displays • Q1: Can computational displays effectively replace glasses in VR/AR? • Q2: How to address the vergence-accommodation conflict for users of different ages? • Q3: What are (in)effective near-eye display technologies? possible solutions: gaze-contingent focus, monovision, light field displays, …
vergence accommodation Conventional Stereo / VR Display
• Visual discomfort (eye tiredness & eyestrain) after ~20 minutes of stereoscopic depth judgments (Hoffman et al. 2008; Shibata et al. 2011) • Degrades visual performance in terms of reaction times and acuity for stereoscopic vision (Hoffman et al. 2008; Konrad et al. 2016; Johnson et al. 2016) Consequences of Vergence-Accommodation Conflict
vergence accommodation Removing VAC with Adaptive Focus
Follow the target with your eyes 4D (0.25m) 0.5D (2m) Task
Stimulus Padmanaban et al., PNAS 2017 Accommodative Response RelativeDistance[D] Time [s]
Stimulus Accommodation n = 59, mean gain = 0.29 Padmanaban et al., PNAS 2017 Accommodative Response RelativeDistance[D] Time [s]
Stimulus Padmanaban et al., PNAS 2017 Accommodative Response RelativeDistance[D] Time [s]
Stimulus Accommodation Padmanaban et al., PNAS 2017 Accommodative Response RelativeDistance[D] Time [s] n = 24, mean gain = 0.77
Duane, 1912 Nearestfocusdistance Age (years) 8 16 24 32 40 48 56 64 72 4D (25cm) 8D (12.5cm) 12D (8cm) Presbyopia 0D (∞cm) 16D (6cm)
Presbyopia
Padmanaban et al., PNAS 2017 Do Presbyopes Benefit from Dynamic Focus? Gain Age
Padmanaban et al., PNAS 2017 Do Presbyopes Benefit from Dynamic Focus? Gain Age conventional
Padmanaban et al., PNAS 2017 Do Presbyopes Benefit from Dynamic Focus? Gain Age conventional dynamic
Padmanaban et al., PNAS 2017 Do Presbyopes Benefit from Dynamic Focus? Gain Age conventional dynamic Response for Physical Stimulus Heron & Charman 2004
far near far near Padmanaban et al., PNAS 2017 Age-dependent FusionPercentFused
far near far near Padmanaban et al., PNAS 2017 Age-dependent FusionPercentFused
far near far near Padmanaban et al., PNAS 2017 Age-dependent FusionPercentFused
far near far near Padmanaban et al., PNAS 2017 Age-dependent SharpnessRelativeSharpness
far near far near Padmanaban et al., PNAS 2017 Age-dependent SharpnessRelativeSharpness
far near far near Padmanaban et al., PNAS 2017 Age-dependent SharpnessRelativeSharpness
• Q1: Can computational displays effectively replace glasses in VR/AR? • Q2: How to address the vergence-accommodation conflict for users of different ages? • Q3: What are (in)effective near-eye display technologies? possible solutions: gaze-contingent focus, monovision, multiplane, light field displays, …
Gaze-contingent Focus • non-presbyopes: adaptive focus is like real world, but needs eye tracking! HMD lens micro display virtual image eye tracking Padmanaban et al., PNAS 2017
Gaze-contingent Focus Padmanaban et al., PNAS 2017
Gaze-contingent Focus Padmanaban et al., PNAS 2017
Gaze-contingent Focus Padmanaban et al., PNAS 2017
at ACM SIGGRAPH 2016
Gaze-contingent Focus – User Preference Padmanaban et al., PNAS 2017
Monovision VR Konrad et al., SIGCHI 2016; Johnson et al., Optics Express 2016; Padmanaban et al., PNAS 2017
Monovision VR Konrad et al., SIGCHI 2016; Johnson et al., Optics Express 2016; Padmanaban et al., PNAS 2017 • monovision did not drive accommodation more than conventional • visually comfortable for most; particularly uncomfortable for some users
Multiplane VR Displays • Rolland J, Krueger M, Goon A (2000) Multifocal planes head-mounted displays. Applied Optics 39 • Akeley K, Watt S, Girshick A, Banks M (2004) A stereo display prototype with multiple focal distances. ACM Trans. Graph. (SIGGRAPH) • Waldkirch M, Lukowicz P, Tröster G (2004) Multiple imaging technique for extending depth of focus in retinal displays. Optics Express • Schowengerdt B, Seibel E (2006) True 3-d scanned voxel displays using single or multiple light sources. JSID • Liu S, Cheng D, Hua H (2008) An optical see-through head mounted display with addressable focal planes in Proc. ISMAR • Love GD et al. (2009) High-speed switchable lens enables the development of a volumetric stereoscopic display. Optics Express • … many more ... idea introduced Rolland et al. 2000 benchtop prototype Akeley 2004 near-eye display prototype Liu 2008, Love 2009
Multiplane VR Displays • Rolland J, Krueger M, Goon A (2000) Multifocal planes head-mounted displays. Applied Optics 39 • Akeley K, Watt S, Girshick A, Banks M (2004) A stereo display prototype with multiple focal distances. ACM Trans. Graph. (SIGGRAPH) • Waldkirch M, Lukowicz P, Tröster G (2004) Multiple imaging technique for extending depth of focus in retinal displays. Optics Express • Schowengerdt B, Seibel E (2006) True 3-d scanned voxel displays using single or multiple light sources. JSID • Liu S, Cheng D, Hua H (2008) An optical see-through head mounted display with addressable focal planes in Proc. ISMAR • Love GD et al. (2009) High-speed switchable lens enables the development of a volumetric stereoscopic display. Optics Express • … many more ... idea introduced Rolland et al. 2000 benchtop prototype Akeley 2004 near-eye display prototype Liu 2008, Love 2009
Light Field CamerasLight Field Stereoscope Huang et al., SIGGRAPH 2015
Backlight Thin Spacer & 2nd panel (6mm) Magnifying Lenses LCD Panel Light Field Stereoscope Huang et al., SIGGRAPH 2015
Near-eye Light Field Displays Idea: project multiple different perspectives into different parts of the pupil!
Target Light Field Input: 4D light field for each eye
Multiplicative Two-layer Modulation Input: 4D light field for each eye
Multiplicative Two-layer Modulation Input: 4D light field for each eye
Multiplicative Two-layer Modulation Input: 4D light field for each eye
Multiplicative Two-layer Modulation Reconstruction: for layer t1 Tensor Displays, Wetzstein et al. 2012 Input: 4D light field for each eye
Traditional HMDs - No Focus Cues The Light Field HMD Stereoscope Light Field Stereoscope Huang et al., SIGGRAPH 2015
Traditional HMDs - No Focus Cues The Light Field HMD Stereoscope Light Field Stereoscope Huang et al., SIGGRAPH 2015
Traditional HMDs - No Focus Cues The Light Field HMD Stereoscope Light Field Stereoscope Huang et al., SIGGRAPH 2015
Traditional HMDs - No Focus Cues The Light Field HMD Stereoscope Light Field Stereoscope Huang et al., SIGGRAPH 2015
Vision-correcting Display iPod Touch prototypeprinted transparency Huang et al., SIGGRAPH 2014
prototype 300 dpi or higher Huang et al., SIGGRAPH 2014
Diffraction in Multilayer Light Field Displays Wetzstein et al., SIGGRAPH 2011 Lanman et al., SIGGRAPH Asia 2011 Wetzstein et al., SIGGRAPH 2012 Maimone et all., Trans. Graph. 2013 … Hirsch et al, SIGGRAPH 2014 No diffraction artifacts with LCoS blur!
Summary • focus cues in VR/AR are challenging • adaptive focus can correct for refractive errors (myopia, hyperopia) • gaze-contingent focus gives natural focus cues for non-presbyopes, but require eyes tracking • presbyopes require fixed focal plane with correction • multiplane displays require very high speed microdisplays • monovision has not demonstrated significant improvements • light field displays may be the “ultimate” display  need to solve “diffraction problem”
Making Virtual Reality Better Than Reality? • focus cues in VR/AR are challenging • adaptive focus can correct for refractive errors (myopia, hyperopia) • gaze-contingent focus gives natural focus cues for non-presbyopes, but require eyes tracking • presbyopes require fixed focal plane with correction, better than reality! • multiplane displays require very high speed microdisplays • monovision has not demonstrated significant improvements • light field displays may be the “ultimate” display  need to solve “diffraction problem”
VR/AR = Frontier of Engineering • Focus cues / visual accessibility • Vestibular-visual conflict (motion sickness) • AR • occlusions • aesthetics / form factor • battery life • heat • wireless operation • low-power computer vision • registration of physical / virtual world and eyes • consistent lighting • scanning real world • VAC more important • display contrast & brightness • fast, embedded GPUs • …
Capturing and Sharing Experiences
It’s Not About Technology but Experiences!
Facebook’s Surround 360 RAW Data: 17 Gb/sec Compute time: days to weeks on conventional computer, minutes to hours on data center
Facebook’s Surround 360 RAW Data: 17 Gb/sec Compute time: days to weeks on conventional computer, minutes to hours on data center
Konrad et al., arxiv 2017
Konrad et al., arxiv 2017
Konrad et al., arxiv 2017
Konrad et al., arxiv 2017
Advancing AR/VR technology requires deep understanding of human vision, optics, signal processing, computation, and more. Technology alone is not enough – engineer experiences! Conclusions
Stanford EE 267
Stanford Computational Imaging Lab Light Field Displays Time-of-Flight Imaging Computational Microscopy Image Optimization Light Field Cameras Near-eye Displays
Stanford Computational Imaging Lab Open Lab this Friday (2/3) 10am-3pm at Stanford Please email Helen Lin (helenlin@stanford.edu) for more details! - "The Light Field Stereoscope" (demo), R. Konrad - "Gaze-contingent and Varifocal Near-eye Displays" (demo), N. Padmanaban - "Monovision Near-eye Displays" (demo), R. Konrad - "Saliency in VR: How do People Explore Virtual Environments?" (poster and demo), V. Sitzmann - "Accommodation-invariant Computational Near-eye Displays" (demo), R. Konrad - "Depth-dependent Visual Anchoring for Reducing Motion Sickness in VR", N. Padmanaban - "ProxImaL: Efficient Image Optimization using Proximal Algorithms" (poster), F. Heide - "Dirty Pixels: Optimizing Image Classification Architectures for Raw Sensor Data" (poster), S. Diamond - "Vortex: Live Cinematic Virtual Reality" (demo), R. Konrad - "Transient Imaging with Single Photon Detectors" (poster), M. O'Toole - "Robust Non-line-of-sight Imaging with Single Photon Detectors" (poster), F. Heide - "Variable Aperture Light Field Photography" (poster), J. Chang - "Computational Time-of-Flight Photography" (poster), F. Heide - "Wide Field-of-View Monocentric Light Field Imaging" (poster and demo), D. Dansereau - "Hacking the Vive Lighthouse - Arduino-based Positional Tracking in VR with Low-cost Components" (demo), K.
Acknowledgements Near-eye Displays • Robert Konrad (Stanford) • Nitish Padmanaban (Stanford) • Fu-Chung Huang (NVIDIA) • Emily Cooper (Dartmouth College) Saliency in VR • Vincent Sitzmann (Stanford) • Diego Gutierrez (U. Zaragoza) • Ana Serrano (U. Zaragoza) • Maneesh Agrawala (Stanford) Spinning VR Camera • Robert Konrad (Stanford) • Donald Dansereau (Stanford) Other • Wolfgang Heidrich (UBC/KAUST) • Ramesh Raskar (MIT/Facebook) • Douglas Lanman (Oculus) • Matt Hirsch (Lumii) • Matthew O’Toole (Stanford) • Felix Heide (Stanford)
Gordon Wetzstein Computational Imaging Lab Stanford University stanford.edu/~gordonwz www.computationalimaging.org

More Related Content

PPTX
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
byStanfordComputationalImaging
 
PPTX
Light Field, Focus-tunable, and Monovision Near-eye Displays | SID 2016
byStanfordComputationalImaging
 
PPTX
Accommodation-invariant Computational Near-eye Displays - SIGGRAPH 2017
byStanfordComputationalImaging
 
PPTX
The Light Field Stereoscope | SIGGRAPH 2015
byStanfordComputationalImaging
 
PPTX
Build Your Own VR Display Course - SIGGRAPH 2017: Part 1
byStanfordComputationalImaging
 
PPTX
Vision-correcting Displays @ SIGGRAPH 2014
byStanfordComputationalImaging
 
PPTX
Tailored Displays to Compensate for Visual Aberrations - SIGGRAPH Presentation
byVitor Pamplona
 
PPTX
Adaptive Spectral Projection
byStanfordComputationalImaging
 
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
byStanfordComputationalImaging
 
Light Field, Focus-tunable, and Monovision Near-eye Displays | SID 2016
byStanfordComputationalImaging
 
Accommodation-invariant Computational Near-eye Displays - SIGGRAPH 2017
byStanfordComputationalImaging
 
The Light Field Stereoscope | SIGGRAPH 2015
byStanfordComputationalImaging
 
Build Your Own VR Display Course - SIGGRAPH 2017: Part 1
byStanfordComputationalImaging
 
Vision-correcting Displays @ SIGGRAPH 2014
byStanfordComputationalImaging
 
Tailored Displays to Compensate for Visual Aberrations - SIGGRAPH Presentation
byVitor Pamplona
 
Adaptive Spectral Projection
byStanfordComputationalImaging
 

What's hot

PPTX
Unusual apps: NETRA and CATRA
byVitor Pamplona
 
PPTX
NETRA on SIGGRAPH 2010
byVitor Pamplona
 
PDF
Gaze-Contingent Ocular Parallax Rendering for Virtual Reality
byStanfordComputationalImaging
 
PDF
Head Mounted Displays: How to realize ultimate AR experiences?
byYutaItoh
 
PDF
Compressive DIsplays: SID Keynote by Ramesh Raskar
byCamera Culture Group, MIT Media Lab
 
PPTX
Stereoscopic Imaging
byCharanjeet Singh
 
PDF
光学シースルーHMDの高性能化に向けたCV技術活用事例
byYuta Itoh
 
PPTX
Google Glass, The META and Co. - How to calibrate your Optical See-Through He...
byJens Grubert
 
PDF
Keynote at VR in Science and Industry
byChristian Sandor
 
PPTX
Compressive Light Field Projection @ SIGGRAPH 2014
byStanfordComputationalImaging
 
PDF
Netra at Mobile Health Expo NYC
byVitor Pamplona
 
PPT
Stereoscopic 3D: Generation Methods and Display Technologies for Industry and...
byRay Phan
 
PPT
CORNAR: Looking Around Corners using Trillion FPS Imaging
byCamera Culture Group, MIT Media Lab
 
PDF
Light Field Technology
byJeffrey Funk
 
PPTX
FocusAR: Auto-focus AR eyeglasses for real world and virtual imagery
byUNC Chapel Hill
 
PPTX
Presentación Tesis 08022016
byUniversidad Politécnica de Madrid
 
PPS
Dyop-Atlan10_Presentation
byAllan Hytowitz
 
PDF
Optical
byeskaparetus
 
DOCX
3D
byrathorenitin87
 
DOCX
3d
byrathorenitin87
 
Unusual apps: NETRA and CATRA
byVitor Pamplona
 
NETRA on SIGGRAPH 2010
byVitor Pamplona
 
Gaze-Contingent Ocular Parallax Rendering for Virtual Reality
byStanfordComputationalImaging
 
Head Mounted Displays: How to realize ultimate AR experiences?
byYutaItoh
 
Compressive DIsplays: SID Keynote by Ramesh Raskar
byCamera Culture Group, MIT Media Lab
 
Stereoscopic Imaging
byCharanjeet Singh
 
光学シースルーHMDの高性能化に向けたCV技術活用事例
byYuta Itoh
 
Google Glass, The META and Co. - How to calibrate your Optical See-Through He...
byJens Grubert
 
Keynote at VR in Science and Industry
byChristian Sandor
 
Compressive Light Field Projection @ SIGGRAPH 2014
byStanfordComputationalImaging
 
Netra at Mobile Health Expo NYC
byVitor Pamplona
 
Stereoscopic 3D: Generation Methods and Display Technologies for Industry and...
byRay Phan
 
CORNAR: Looking Around Corners using Trillion FPS Imaging
byCamera Culture Group, MIT Media Lab
 
Light Field Technology
byJeffrey Funk
 
FocusAR: Auto-focus AR eyeglasses for real world and virtual imagery
byUNC Chapel Hill
 
Presentación Tesis 08022016
byUniversidad Politécnica de Madrid
 
Dyop-Atlan10_Presentation
byAllan Hytowitz
 
Optical
byeskaparetus
 
3D
byrathorenitin87
 
3d
byrathorenitin87
 

Viewers also liked

PPTX
Multi-camera Time-of-Flight Systems | SIGGRAPH 2016
byStanfordComputationalImaging
 
PPTX
ProxImaL | SIGGRAPH 2016
byStanfordComputationalImaging
 
PDF
Intro to ETW Tracing and OSVR
byRyan A. Pavlik
 
PPTX
Keynote - SPIE Stereoscopic Displays & Applications 2014
byGordon Wetzstein
 
PPTX
SIGGRAPH 2012 Computational Plenoptic Imaging Course - 2 High Dynamic Range I...
byGordon Wetzstein
 
PPTX
SIGGRAPH 2012 Computational Plenoptic Imaging Course - 1 Introduction
byGordon Wetzstein
 
PPTX
Compressive Light Field Displays
byGordon Wetzstein
 
PPTX
SIGGRAPH 2012 Computational Plenoptic Imaging Course - 7 Schlieren Imaging
byGordon Wetzstein
 
PPTX
SIGGRAPH 2012 Computational Plenoptic Imaging Course - 4 Light Fields
byGordon Wetzstein
 
PPTX
SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging
byGordon Wetzstein
 
PDF
130935887 areva-kvfg-r8559 c
byJuan Carlos Valderrama
 
Multi-camera Time-of-Flight Systems | SIGGRAPH 2016
byStanfordComputationalImaging
 
ProxImaL | SIGGRAPH 2016
byStanfordComputationalImaging
 
Intro to ETW Tracing and OSVR
byRyan A. Pavlik
 
Keynote - SPIE Stereoscopic Displays & Applications 2014
byGordon Wetzstein
 
SIGGRAPH 2012 Computational Plenoptic Imaging Course - 2 High Dynamic Range I...
byGordon Wetzstein
 
SIGGRAPH 2012 Computational Plenoptic Imaging Course - 1 Introduction
byGordon Wetzstein
 
Compressive Light Field Displays
byGordon Wetzstein
 
SIGGRAPH 2012 Computational Plenoptic Imaging Course - 7 Schlieren Imaging
byGordon Wetzstein
 
SIGGRAPH 2012 Computational Plenoptic Imaging Course - 4 Light Fields
byGordon Wetzstein
 
SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging
byGordon Wetzstein
 
130935887 areva-kvfg-r8559 c
byJuan Carlos Valderrama
 

Similar to VR2.0: Making Virtual Reality Better Than Reality?

PDF
Augmented Reality in Volumetric Medical Imaging Using Stereoscopic 3D Display
byijcga
 
PDF
Comp4010 lecture3-AR Technology
byMark Billinghurst
 
PDF
COMP 4010 - Lecture4 VR Technology - Visual and Haptic Displays
byMark Billinghurst
 
PDF
Keynote at 23rd International Display Workshop
byChristian Sandor
 
PPTX
Binocular Eye Tracking and Calibration in Head-mounted Displays
byMichael Stengel
 
PDF
Adlens SXSW 2020 proposal presentation
byPrashanthan Ganeswaran
 
PDF
A UGMENT R EALITY IN V OLUMETRIC M EDICAL I MAGING U SING S TEREOSCOPIC...
byijcga
 
PDF
Future Directions for Augmented Reality
byMark Billinghurst
 
PDF
Comp4010 Lecture12 Research Directions
byMark Billinghurst
 
PPTX
Emerging 3D Display Technologies
byMatt Hirsch - MIT Media Lab
 
PDF
Introduction to Optical See-Through HMDs in AR
byYuta Itoh
 
PDF
Real-Time Simulation of Impaired Vision in Naturalistic Settings with Gaze-Co...
byMargarita Vinnikov
 
PDF
Eyeglasses-free Display: Towards Correcting Visual Aberrations with Computati...
byDario Caliendo
 
PDF
Sig2014 vision correcting display
byWael Sharba
 
PPTX
Orkun Oguz (CyVision): Critical Factors for the Mass Adaption of XR
byAugmentedWorldExpo
 
PDF
COMP 4010 Lecture3: Human Perception
byMark Billinghurst
 
PDF
Virtual Reality: Creating Accessible Virtual Worlds
byColson Rice
 
PDF
Augmented reality
bygetaugustine
 
PPTX
Depth & space
bybecca_kennedy
 
PDF
“Eye Tracking for the Future,” a Presentation from Parallel Rules
byEdge AI and Vision Alliance
 
Augmented Reality in Volumetric Medical Imaging Using Stereoscopic 3D Display
byijcga
 
Comp4010 lecture3-AR Technology
byMark Billinghurst
 
COMP 4010 - Lecture4 VR Technology - Visual and Haptic Displays
byMark Billinghurst
 
Keynote at 23rd International Display Workshop
byChristian Sandor
 
Binocular Eye Tracking and Calibration in Head-mounted Displays
byMichael Stengel
 
Adlens SXSW 2020 proposal presentation
byPrashanthan Ganeswaran
 
A UGMENT R EALITY IN V OLUMETRIC M EDICAL I MAGING U SING S TEREOSCOPIC...
byijcga
 
Future Directions for Augmented Reality
byMark Billinghurst
 
Comp4010 Lecture12 Research Directions
byMark Billinghurst
 
Emerging 3D Display Technologies
byMatt Hirsch - MIT Media Lab
 
Introduction to Optical See-Through HMDs in AR
byYuta Itoh
 
Real-Time Simulation of Impaired Vision in Naturalistic Settings with Gaze-Co...
byMargarita Vinnikov
 
Eyeglasses-free Display: Towards Correcting Visual Aberrations with Computati...
byDario Caliendo
 
Sig2014 vision correcting display
byWael Sharba
 
Orkun Oguz (CyVision): Critical Factors for the Mass Adaption of XR
byAugmentedWorldExpo
 
COMP 4010 Lecture3: Human Perception
byMark Billinghurst
 
Virtual Reality: Creating Accessible Virtual Worlds
byColson Rice
 
Augmented reality
bygetaugustine
 
Depth & space
bybecca_kennedy
 
“Eye Tracking for the Future,” a Presentation from Parallel Rules
byEdge AI and Vision Alliance
 

More from StanfordComputationalImaging

PPTX
Autofocals: Evaluating Gaze-Contingent Eyeglasses for Presbyopes - Siggraph 2019
byStanfordComputationalImaging
 
PPTX
Non-line-of-sight Imaging with Partial Occluders and Surface Normals | TOG 2019
byStanfordComputationalImaging
 
PPTX
End-to-end Optimization of Cameras and Image Processing - SIGGRAPH 2018
byStanfordComputationalImaging
 
PPTX
Build Your Own VR Display Course - SIGGRAPH 2017: Part 5
byStanfordComputationalImaging
 
PPTX
Build Your Own VR Display Course - SIGGRAPH 2017: Part 4
byStanfordComputationalImaging
 
PPTX
Build Your Own VR Display Course - SIGGRAPH 2017: Part 3
byStanfordComputationalImaging
 
PPTX
Build Your Own VR Display Course - SIGGRAPH 2017: Part 2
byStanfordComputationalImaging
 
Autofocals: Evaluating Gaze-Contingent Eyeglasses for Presbyopes - Siggraph 2019
byStanfordComputationalImaging
 
Non-line-of-sight Imaging with Partial Occluders and Surface Normals | TOG 2019
byStanfordComputationalImaging
 
End-to-end Optimization of Cameras and Image Processing - SIGGRAPH 2018
byStanfordComputationalImaging
 
Build Your Own VR Display Course - SIGGRAPH 2017: Part 5
byStanfordComputationalImaging
 
Build Your Own VR Display Course - SIGGRAPH 2017: Part 4
byStanfordComputationalImaging
 
Build Your Own VR Display Course - SIGGRAPH 2017: Part 3
byStanfordComputationalImaging
 
Build Your Own VR Display Course - SIGGRAPH 2017: Part 2
byStanfordComputationalImaging
 

Recently uploaded

PDF
Rajesh Prasad- Brief Profile with educational, professional highlights
byRajesh Prasad
 
PDF
engineering management chapter 5 ppt presentation
byericaangelatoledo06
 
PDF
Shear Strength of Soil (Direct shear test).pdf
byMahmoodKhalid11
 
PDF
Enhancing optical fiber communication systems using repetition coding for imp...
byMouweffeqBouregaa
 
PPTX
Presentation-WPS Office.pptx afgouvhyhgfccf
byEndaleTimerga
 
PDF
AWS Re:Invent 2025 Recap by FivexL - Guilherme, Vladimir, Andrey
byAndrey Devyatkin
 
PPTX
A professional presentation on Cosmos Bank Heist
byssuser7f0075
 
PDF
Weak Incentives (WINK): The Agent Definition Layer
byAndrei Savu
 
PDF
Unit-I Process Management 1.9 Scheduling Criteria, Scheduling Algorithms
bySanjay Gunjal
 
PDF
Decision-Support-Systems-and-Decision-Making-Processes.pdf
byPatankarNikhil
 
PPTX
Value engineering and cost analysis with case study
byhp9879098082
 
PPT
Integer , Goal and Non linear Programming Model
byOcheriCyril2
 
PPTX
Fitting Infiltration Models to Infiltration using Excel (1).pptx
byTomNickoRivera1
 
PPTX
traffic safety section seven (Traffic Control and Management) of Act
byazeRakeshSunariMagar
 
PPT
Oracle Advanced subquery and types of subquery
bytejaswinikulkarni549
 
PPTX
Fuel Injection Pump Test Bench – Precision Testing & Calibration for Diesel E...
byNeometrix_Engineering_Pvt_Ltd
 
PPTX
How Does LNG Regasification Work | INOXCVA
byINOXCVA
 
PDF
Infinite Sequence and Series: It Includes basic Sequence and Series
byDynamicDomain1
 
PPTX
GET 211- Computing and Software Engineering (1).pptx
byfestusdaniel142
 
PDF
Basics of Electronics Task by Vivaan Jo Varghese.pdf
byVivaan Jo Varghese
 
Rajesh Prasad- Brief Profile with educational, professional highlights
byRajesh Prasad
 
engineering management chapter 5 ppt presentation
byericaangelatoledo06
 
Shear Strength of Soil (Direct shear test).pdf
byMahmoodKhalid11
 
Enhancing optical fiber communication systems using repetition coding for imp...
byMouweffeqBouregaa
 
Presentation-WPS Office.pptx afgouvhyhgfccf
byEndaleTimerga
 
AWS Re:Invent 2025 Recap by FivexL - Guilherme, Vladimir, Andrey
byAndrey Devyatkin
 
A professional presentation on Cosmos Bank Heist
byssuser7f0075
 
Weak Incentives (WINK): The Agent Definition Layer
byAndrei Savu
 
Unit-I Process Management 1.9 Scheduling Criteria, Scheduling Algorithms
bySanjay Gunjal
 
Decision-Support-Systems-and-Decision-Making-Processes.pdf
byPatankarNikhil
 
Value engineering and cost analysis with case study
byhp9879098082
 
Integer , Goal and Non linear Programming Model
byOcheriCyril2
 
Fitting Infiltration Models to Infiltration using Excel (1).pptx
byTomNickoRivera1
 
traffic safety section seven (Traffic Control and Management) of Act
byazeRakeshSunariMagar
 
Oracle Advanced subquery and types of subquery
bytejaswinikulkarni549
 
Fuel Injection Pump Test Bench – Precision Testing & Calibration for Diesel E...
byNeometrix_Engineering_Pvt_Ltd
 
How Does LNG Regasification Work | INOXCVA
byINOXCVA
 
Infinite Sequence and Series: It Includes basic Sequence and Series
byDynamicDomain1
 
GET 211- Computing and Software Engineering (1).pptx
byfestusdaniel142
 
Basics of Electronics Task by Vivaan Jo Varghese.pdf
byVivaan Jo Varghese
 

VR2.0: Making Virtual Reality Better Than Reality?

  • 1.
    Making Virtual Realitybetter than Reality? Gordon Wetzstein Stanford University IS&T Electronic Imaging 2017 www.computationalimaging.org
  • 3.
    Personal Computer e.g. CommodorePET 1983 Laptop e.g. Apple MacBook Smartphone e.g. Google Pixel AR/VR e.g. Microsoft Hololens ???
  • 4.
    A Brief Historyof Virtual Reality 1838 1968 2012-2017 Stereoscopes Wheatstone, Brewster, … VR & AR Ivan Sutherland VR explosion Oculus, Sony, HTC, MS, … Nintendo Virtual Boy 1995 VR 2.0
  • 5.
    Where we arenow IFIXIT teardown
  • 8.
  • 15.
  • 16.
    Current VR Displays: Vergence& Accommodation Mismatch for people with normal vision
  • 17.
    Presbyopia [Katz et al.1997] 68% age 80+ 43% age 40 25% Hyperopia [Krachmer et al. 2005] Myopia 41.6% [Vitale et al. 2009] How Many People Have Normal Vision? all numbers of US population
  • 18.
    4D / 25cmOptical Infinity Normal vision Nearsighted/myopic Farsighted/Hyperopic Presbyopic Focal range (range of clear vision) Modified from Pamplona et al, Proc. of SIGGRAPH 2010 Nearsightedness & Farsightedness
  • 19.
  • 20.
    • Q1: Cancomputational displays effectively replace glasses in VR/AR? • Q2: How to address the vergence-accommodation conflict for users of different ages? • Q3: What are (in)effective near-eye display technologies? possible solutions: gaze-contingent focus, monovision, multiplane, light field displays, …
  • 21.
    • Q1: Cancomputational displays effectively replace glasses in VR/AR? • Q2: How to address the vergence-accommodation conflict for users of different ages? • Q3: What are (in)effective near-eye display technologies? possible solutions: gaze-contingent focus, monovision, multiplane, light field displays, …
  • 22.
  • 23.
  • 24.
  • 25.
    Adaptive Focus -History • M. Heilig “Sensorama”, 1962 (US Patent #3,050,870) • P. Mills, H. Fuchs, S. Pizer “High-Speed Interaction On A Vibrating-Mirror 3D Display”, SPIE 0507 1984 • S. Shiwa, K. Omura, F. Kishino “Proposal for a 3-D display with accommodative compensation: 3DDAC”, JSID 1996 • S. McQuaide, E. Seibel, J. Kelly, B. Schowengerdt, T. Furness “A retinal scanning display system that produces multiple focal planes with a deformable membrane mirror”, Displays 2003 • S. Liu, D. Cheng, H. Hua “An optical see-through head mounted display with addressable focal planes”, Proc. ISMAR 2008 manual focus adjustment Heilig 1962 automatic focus adjustment Mills 1984 deformabe mirrors & lenses McQuaide 2003, Liu 2008
  • 26.
  • 27.
  • 28.
  • 29.
  • 30.
  • 31.
  • 32.
  • 36.
    at ACM SIGGRAPH2016 EyeNetra.com
  • 37.
    at ACM SIGGRAPH2016 participants of the study, 152 total EyeNetra.com
  • 38.
    Participants - Prescription Padmanabanet al., PNAS 2017 n = 70, ages 21-64
  • 39.
    How sharp isthe target? (blurry, medium, sharp) Is the target fused? (yes, no) 4D (0.25m) 3D (0.33m) 2D (0.50m) 1D (1m) Four simulated distances Task
  • 40.
    far near 1D 1m 2D 0.5m 3D 0.3m 4D 0.25m Distance medium 0 Relativesharpness sharp1 blurry -1 VR uncorrected VR corrected Results - Sharpness Padmanaban et al., PNAS 2017
  • 41.
    far near far near 1D 1m 2D 0.5m 3D 0.3m 4D 0.25m Distance medium0 sharp 1 blurry -1 Relativesharpness VR uncorrected VR corrected Results - Sharpness Padmanaban et al., PNAS 2017
  • 42.
    far near far near 1D 1m 2D 0.5m 3D 0.3m 4D 0.25m Distance medium0 sharp 1 blurry -1 Relativesharpness VR uncorrected VR corrected Results - Sharpness Padmanaban et al., PNAS 2017
  • 43.
    Mean = 0.63 Mean= 0.60 far near far near 1D 1m 2D 0.5m 3D 0.3m 4D 0.25m Distance medium 0 sharp 1 blurry -1 Relativesharpness VR uncorrected VR corrected normal correction Results - Sharpness Padmanaban et al., PNAS 2017
  • 44.
    far far near 1D 1m 2D 0.5m 3D 0.3m 4D 0.25m Distance VR uncorrected VRcorrected 1 Proportionfused 0.8 0.6 0.4 0.2 0 Results - Fusion Padmanaban et al., PNAS 2017
  • 45.
    far far near 1D 1m 2D 0.5m 3D 0.3m 4D 0.25m Distance VR uncorrected VRcorrected 1 Proportionfused 0.8 0.6 0.4 0.2 0 Results - Fusion Padmanaban et al., PNAS 2017
  • 46.
    Computational Near-eye Displays •Q1: Can computational displays effectively replace glasses in VR/AR? • Q2: How to address the vergence-accommodation conflict for users of different ages? • Q3: What are (in)effective near-eye display technologies? possible solutions: gaze-contingent focus, monovision, light field displays, …
  • 47.
  • 48.
    • Visual discomfort(eye tiredness & eyestrain) after ~20 minutes of stereoscopic depth judgments (Hoffman et al. 2008; Shibata et al. 2011) • Degrades visual performance in terms of reaction times and acuity for stereoscopic vision (Hoffman et al. 2008; Konrad et al. 2016; Johnson et al. 2016) Consequences of Vergence-Accommodation Conflict
  • 49.
  • 50.
    Follow the targetwith your eyes 4D (0.25m) 0.5D (2m) Task
  • 51.
    Stimulus Padmanaban et al.,PNAS 2017 Accommodative Response RelativeDistance[D] Time [s]
  • 52.
    Stimulus Accommodation n = 59,mean gain = 0.29 Padmanaban et al., PNAS 2017 Accommodative Response RelativeDistance[D] Time [s]
  • 53.
    Stimulus Padmanaban et al.,PNAS 2017 Accommodative Response RelativeDistance[D] Time [s]
  • 54.
    Stimulus Accommodation Padmanaban et al.,PNAS 2017 Accommodative Response RelativeDistance[D] Time [s] n = 24, mean gain = 0.77
  • 55.
    Duane, 1912 Nearestfocusdistance Age (years) 816 24 32 40 48 56 64 72 4D (25cm) 8D (12.5cm) 12D (8cm) Presbyopia 0D (∞cm) 16D (6cm)
  • 56.
  • 57.
    Padmanaban et al.,PNAS 2017 Do Presbyopes Benefit from Dynamic Focus? Gain Age
  • 58.
    Padmanaban et al.,PNAS 2017 Do Presbyopes Benefit from Dynamic Focus? Gain Age conventional
  • 59.
    Padmanaban et al.,PNAS 2017 Do Presbyopes Benefit from Dynamic Focus? Gain Age conventional dynamic
  • 60.
    Padmanaban et al.,PNAS 2017 Do Presbyopes Benefit from Dynamic Focus? Gain Age conventional dynamic Response for Physical Stimulus Heron & Charman 2004
  • 61.
    far near farnear Padmanaban et al., PNAS 2017 Age-dependent FusionPercentFused
  • 62.
    far near farnear Padmanaban et al., PNAS 2017 Age-dependent FusionPercentFused
  • 63.
    far near farnear Padmanaban et al., PNAS 2017 Age-dependent FusionPercentFused
  • 64.
    far near farnear Padmanaban et al., PNAS 2017 Age-dependent SharpnessRelativeSharpness
  • 65.
    far near farnear Padmanaban et al., PNAS 2017 Age-dependent SharpnessRelativeSharpness
  • 66.
    far near farnear Padmanaban et al., PNAS 2017 Age-dependent SharpnessRelativeSharpness
  • 67.
    • Q1: Cancomputational displays effectively replace glasses in VR/AR? • Q2: How to address the vergence-accommodation conflict for users of different ages? • Q3: What are (in)effective near-eye display technologies? possible solutions: gaze-contingent focus, monovision, multiplane, light field displays, …
  • 68.
    Gaze-contingent Focus • non-presbyopes:adaptive focus is like real world, but needs eye tracking! HMD lens micro display virtual image eye tracking Padmanaban et al., PNAS 2017
  • 69.
  • 70.
  • 71.
  • 72.
  • 73.
    Gaze-contingent Focus –User Preference Padmanaban et al., PNAS 2017
  • 74.
    Monovision VR Konrad etal., SIGCHI 2016; Johnson et al., Optics Express 2016; Padmanaban et al., PNAS 2017
  • 75.
    Monovision VR Konrad etal., SIGCHI 2016; Johnson et al., Optics Express 2016; Padmanaban et al., PNAS 2017 • monovision did not drive accommodation more than conventional • visually comfortable for most; particularly uncomfortable for some users
  • 76.
    Multiplane VR Displays •Rolland J, Krueger M, Goon A (2000) Multifocal planes head-mounted displays. Applied Optics 39 • Akeley K, Watt S, Girshick A, Banks M (2004) A stereo display prototype with multiple focal distances. ACM Trans. Graph. (SIGGRAPH) • Waldkirch M, Lukowicz P, Tröster G (2004) Multiple imaging technique for extending depth of focus in retinal displays. Optics Express • Schowengerdt B, Seibel E (2006) True 3-d scanned voxel displays using single or multiple light sources. JSID • Liu S, Cheng D, Hua H (2008) An optical see-through head mounted display with addressable focal planes in Proc. ISMAR • Love GD et al. (2009) High-speed switchable lens enables the development of a volumetric stereoscopic display. Optics Express • … many more ... idea introduced Rolland et al. 2000 benchtop prototype Akeley 2004 near-eye display prototype Liu 2008, Love 2009
  • 77.
    Multiplane VR Displays •Rolland J, Krueger M, Goon A (2000) Multifocal planes head-mounted displays. Applied Optics 39 • Akeley K, Watt S, Girshick A, Banks M (2004) A stereo display prototype with multiple focal distances. ACM Trans. Graph. (SIGGRAPH) • Waldkirch M, Lukowicz P, Tröster G (2004) Multiple imaging technique for extending depth of focus in retinal displays. Optics Express • Schowengerdt B, Seibel E (2006) True 3-d scanned voxel displays using single or multiple light sources. JSID • Liu S, Cheng D, Hua H (2008) An optical see-through head mounted display with addressable focal planes in Proc. ISMAR • Love GD et al. (2009) High-speed switchable lens enables the development of a volumetric stereoscopic display. Optics Express • … many more ... idea introduced Rolland et al. 2000 benchtop prototype Akeley 2004 near-eye display prototype Liu 2008, Love 2009
  • 78.
    Light Field CamerasLightField Stereoscope Huang et al., SIGGRAPH 2015
  • 79.
    Backlight Thin Spacer &2nd panel (6mm) Magnifying Lenses LCD Panel Light Field Stereoscope Huang et al., SIGGRAPH 2015
  • 80.
    Near-eye Light FieldDisplays Idea: project multiple different perspectives into different parts of the pupil!
  • 81.
    Target Light Field Input:4D light field for each eye
  • 82.
    Multiplicative Two-layer ModulationInput: 4D light field for each eye
  • 83.
    Multiplicative Two-layer ModulationInput: 4D light field for each eye
  • 84.
    Multiplicative Two-layer ModulationInput: 4D light field for each eye
  • 85.
    Multiplicative Two-layer Modulation Reconstruction: forlayer t1 Tensor Displays, Wetzstein et al. 2012 Input: 4D light field for each eye
  • 86.
    Traditional HMDs - NoFocus Cues The Light Field HMD Stereoscope Light Field Stereoscope Huang et al., SIGGRAPH 2015
  • 87.
    Traditional HMDs - NoFocus Cues The Light Field HMD Stereoscope Light Field Stereoscope Huang et al., SIGGRAPH 2015
  • 88.
    Traditional HMDs - NoFocus Cues The Light Field HMD Stereoscope Light Field Stereoscope Huang et al., SIGGRAPH 2015
  • 89.
    Traditional HMDs - NoFocus Cues The Light Field HMD Stereoscope Light Field Stereoscope Huang et al., SIGGRAPH 2015
  • 90.
    Vision-correcting Display iPod Touchprototypeprinted transparency Huang et al., SIGGRAPH 2014
  • 91.
    prototype 300 dpi orhigher Huang et al., SIGGRAPH 2014
  • 92.
    Diffraction in MultilayerLight Field Displays Wetzstein et al., SIGGRAPH 2011 Lanman et al., SIGGRAPH Asia 2011 Wetzstein et al., SIGGRAPH 2012 Maimone et all., Trans. Graph. 2013 … Hirsch et al, SIGGRAPH 2014 No diffraction artifacts with LCoS blur!
  • 93.
    Summary • focus cuesin VR/AR are challenging • adaptive focus can correct for refractive errors (myopia, hyperopia) • gaze-contingent focus gives natural focus cues for non-presbyopes, but require eyes tracking • presbyopes require fixed focal plane with correction • multiplane displays require very high speed microdisplays • monovision has not demonstrated significant improvements • light field displays may be the “ultimate” display  need to solve “diffraction problem”
  • 94.
    Making Virtual RealityBetter Than Reality? • focus cues in VR/AR are challenging • adaptive focus can correct for refractive errors (myopia, hyperopia) • gaze-contingent focus gives natural focus cues for non-presbyopes, but require eyes tracking • presbyopes require fixed focal plane with correction, better than reality! • multiplane displays require very high speed microdisplays • monovision has not demonstrated significant improvements • light field displays may be the “ultimate” display  need to solve “diffraction problem”
  • 95.
    VR/AR = Frontierof Engineering • Focus cues / visual accessibility • Vestibular-visual conflict (motion sickness) • AR • occlusions • aesthetics / form factor • battery life • heat • wireless operation • low-power computer vision • registration of physical / virtual world and eyes • consistent lighting • scanning real world • VAC more important • display contrast & brightness • fast, embedded GPUs • …
  • 96.
  • 97.
    It’s Not AboutTechnology but Experiences!
  • 103.
    Facebook’s Surround 360 RAWData: 17 Gb/sec Compute time: days to weeks on conventional computer, minutes to hours on data center
  • 104.
    Facebook’s Surround 360 RAWData: 17 Gb/sec Compute time: days to weeks on conventional computer, minutes to hours on data center
  • 105.
    Konrad et al.,arxiv 2017
  • 106.
    Konrad et al.,arxiv 2017
  • 107.
    Konrad et al.,arxiv 2017
  • 108.
    Konrad et al.,arxiv 2017
  • 109.
    Advancing AR/VR technologyrequires deep understanding of human vision, optics, signal processing, computation, and more. Technology alone is not enough – engineer experiences! Conclusions
  • 110.
  • 111.
    Stanford Computational ImagingLab Light Field Displays Time-of-Flight Imaging Computational Microscopy Image Optimization Light Field Cameras Near-eye Displays
  • 112.
    Stanford Computational ImagingLab Open Lab this Friday (2/3) 10am-3pm at Stanford Please email Helen Lin (helenlin@stanford.edu) for more details! - "The Light Field Stereoscope" (demo), R. Konrad - "Gaze-contingent and Varifocal Near-eye Displays" (demo), N. Padmanaban - "Monovision Near-eye Displays" (demo), R. Konrad - "Saliency in VR: How do People Explore Virtual Environments?" (poster and demo), V. Sitzmann - "Accommodation-invariant Computational Near-eye Displays" (demo), R. Konrad - "Depth-dependent Visual Anchoring for Reducing Motion Sickness in VR", N. Padmanaban - "ProxImaL: Efficient Image Optimization using Proximal Algorithms" (poster), F. Heide - "Dirty Pixels: Optimizing Image Classification Architectures for Raw Sensor Data" (poster), S. Diamond - "Vortex: Live Cinematic Virtual Reality" (demo), R. Konrad - "Transient Imaging with Single Photon Detectors" (poster), M. O'Toole - "Robust Non-line-of-sight Imaging with Single Photon Detectors" (poster), F. Heide - "Variable Aperture Light Field Photography" (poster), J. Chang - "Computational Time-of-Flight Photography" (poster), F. Heide - "Wide Field-of-View Monocentric Light Field Imaging" (poster and demo), D. Dansereau - "Hacking the Vive Lighthouse - Arduino-based Positional Tracking in VR with Low-cost Components" (demo), K.
  • 113.
    Acknowledgements Near-eye Displays • RobertKonrad (Stanford) • Nitish Padmanaban (Stanford) • Fu-Chung Huang (NVIDIA) • Emily Cooper (Dartmouth College) Saliency in VR • Vincent Sitzmann (Stanford) • Diego Gutierrez (U. Zaragoza) • Ana Serrano (U. Zaragoza) • Maneesh Agrawala (Stanford) Spinning VR Camera • Robert Konrad (Stanford) • Donald Dansereau (Stanford) Other • Wolfgang Heidrich (UBC/KAUST) • Ramesh Raskar (MIT/Facebook) • Douglas Lanman (Oculus) • Matt Hirsch (Lumii) • Matthew O’Toole (Stanford) • Felix Heide (Stanford)
  • 114.
    Gordon Wetzstein Computational ImagingLab Stanford University stanford.edu/~gordonwz www.computationalimaging.org