Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
The Light Field Stereoscope:
Immersive Computer Graphics via Factored
Near-Eye Light Field Displays with Focus Cues
Fu-Chu...
Top View
Vergence & Accommodation Match!
Left Eye Right Eye
(Rotation) (Focus)
RealWorld
Parallax
Over Pupil
Top View
HMD
Screen
Stereo
HMD
Vergence & Accommodation Mismatch!
Left Eye Right Eye
RealWorld
No Parallax!
Parallax
Over ...
Lenses
Displays
Drivers
…
…
…
$20
$70
$60
On ebay:
DIY Light Field Stereoscope
Gaming
Da vinci surgical system
HEALTH and SAFETY WARNING
HEALTH and SAFETY WARNING
focus at the object’s distance
Vergence & accommodation are coupled!
Vergence & accommodation are coupled!
Immersive HMD Magnified Display
Immersive HMD Magnified Display
Decoupled!
Immersive HMD Magnified Display
!@#$$^R$%%&$
discomfort and fatigue, eyestrain, diplopic vision, headaches, nau...
0.001
0.01
0.1
1
1m 10m 100m 1000m 10000m
Personal Action Vista
Relative Height
Arial
Perspective
DepthContrast
Distance [...
Backlight
Thin Spacer & 2nd panel
(6mm)
Magnifying Lenses
LCD Panel
Front Focusing
Rear Focusing
Stereoscope
Wheatstone, Brewster
1838
VR/AR
Ivan Sutherland
1968
VR explosion
Oculus, Sony, Valve, MS, …
Now Future
Future...
Stereoscope
Wheatstone, Brewster
1838
VR/AR
Ivan Sutherland
1968
VR explosion
Oculus, Sony, Valve, MS, …
Now Future
Future...
Object emits natural light rays
Light field enters eyes
Layered3D, Tensor Display
[Wetzstein et al. 2011, 12]
Small Angle
Stationary
TV vs. HMD
• Reduced brightness problematic
• Shared: Need wide viewing angle
– High Rank
– Multiple Layers
– Temporal Mul...
Low-rank, 2-panel
No temporal multiplexing
Both binocular & focus cues
to form 3D perception
Low-rank, 2-panel
No temporal multiplexing
Target Light Field
Input: 4D light field for each eye
Multiplicative Two-layer Modulation
1
𝑙 𝑥, 𝑣
𝑥
𝑣
𝑡1
𝑡2
𝑙 𝑥, 𝑣 = 𝑡1 𝑥 𝑡2 𝑥 −
(𝑣 − 𝑥)
𝑑 𝑒
𝑑 𝑒
Input: 4D light field for each...
Multiplicative Two-layer Modulation
1
𝑡1
𝑡2
𝑙 𝑥, 𝑣1 = 𝑡1 𝑥 𝑡2 𝑥 −
(𝑣1 − 𝑥)
𝑑 𝑒
𝑑 𝑒
Input: 4D light field for each eye
Multiplicative Two-layer Modulation
1
𝑡1
𝑡2
𝑙 𝑥, 𝑣4 = 𝑡1 𝑥 𝑡2 𝑥 −
(𝑣4 − 𝑥)
𝑑 𝑒
𝑑 𝑒
Input: 4D light field for each eye
Multiplicative Two-layer Modulation
1
𝑡1
𝑡2
𝑙 𝑥, 𝑣7 = 𝑡1 𝑥 𝑡2 𝑥 −
(𝑣7 − 𝑥)
𝑑 𝑒
𝑑 𝑒
Input: 4D light field for each eye
Multiplicative Two-layer Modulation
𝑡1
𝑡2
Input: 4D light field for each eye
Parallax over the Pupil
Multiplicative Two-layer Modulation
𝑡1
𝑡2
l = (ϕ1 𝑡1)o(ϕ2 𝑡2)Reconstruction:
Input: 4D light field for each eye
𝑚𝑖𝑛𝑖𝑚𝑖𝑧𝑒 𝛽...
Input light field to the eyeMultiplicative Two-layer Modulation
𝑡1
𝑡2
Left Eye: Front Panel Left Eye: Rear Panel
Front Pan...
Front Focus Mid Focus Rear Focus
Front Object Mid Object Rear Object
Multiplicative Two-layer Modulation
𝑡1
𝑡2
Front Panel...
Multilayer Displays
Akeley et al. [2004]
Love et al. [2009]
Narain et al.[2015]
Optical
Overlay
Temporal
Multiplexing
Central Light Field
Right Light Field
Left Light Field
With Additive Multilayer
Left Light Field
Additive
Zoom-In
Multiplicative
The Hardware
- Adafruit based 3D printed housing
learn.adafruit.com/3d-printed-wearable
-video-goggles
- Aspheric lens, 2”...
E-Tech Demo
South Hall G, Booth ET18
The Hardware
Content Generation
• 5x5x2 Stereo Light Field
• GPU Factorization
• 3 ~ 5 Iterations per frame
• 25(views) x 640x800 x 2(e...
Photograph Results
Traditional HMDs
- No Focus Cues
The Light Field HMD
Stereoscope
Photograph Results
Traditional HMDs
- No Focus Cues
The Light Field HMD
Stereoscope
Photograph Results
Traditional HMDs
- No Focus Cues
The Light Field HMD
Stereoscope
Photograph Results
Traditional HMDs
- No Focus Cues
The Light Field HMD
Stereoscope
Photograph Results
Traditional HMDs
- No Focus Cues
The Light Field HMD
Stereoscope
Photograph Results
Traditional HMDs
- No Focus Cues
The Light Field HMD
Stereoscope
Photograph Results
Traditional HMDs
- No Focus Cues
The Light Field HMD
Stereoscope
Photograph Results
Traditional HMDs
- No Focus Cues
The Light Field HMD
Stereoscope
Designing Real-world Experience
Left eye Right eye
Front
Focusing
Rear
Focusing
Technical Details in the Paper
Good Parameter
Space
Limited by Diffraction
100 200 300 400 500
2
3
5
6
4
Front Panel PPI
N...
Discussion
• Reduction in brightness
• Latency: 5x5x2 views
• Diffraction
Future Work
Light Field Content Human Vision/Perception
IPD / 2 * M2
IPD / 2 * M1
Center
Center
Top View Light Field Side View Light Field
The Light Field Stereoscope | SIGGRAPH 2015
The Light Field Stereoscope | SIGGRAPH 2015
The Light Field Stereoscope | SIGGRAPH 2015
The Light Field Stereoscope | SIGGRAPH 2015
The Light Field Stereoscope | SIGGRAPH 2015
The Light Field Stereoscope | SIGGRAPH 2015
The Light Field Stereoscope | SIGGRAPH 2015
The Light Field Stereoscope | SIGGRAPH 2015
The Light Field Stereoscope | SIGGRAPH 2015
The Light Field Stereoscope | SIGGRAPH 2015
Upcoming SlideShare
Loading in …5
×

The Light Field Stereoscope | SIGGRAPH 2015

24,555 views

Published on

Inspired by Wheatstone’s original stereoscope and augmenting it with modern factored light field synthesis, we present a new near-eye display technology that supports focus cues. These cues are critical for mitigating visual discomfort experienced in commercially-available head mounted displays and providing comfortable, long-term immersive experiences.

Published in: Technology
  • Be the first to comment

The Light Field Stereoscope | SIGGRAPH 2015

  1. 1. The Light Field Stereoscope: Immersive Computer Graphics via Factored Near-Eye Light Field Displays with Focus Cues Fu-Chung Huang1,2 Kevin Chen1 Gordon Wetzstein1 1Stanford University 2Now at NVIDIA Research
  2. 2. Top View Vergence & Accommodation Match! Left Eye Right Eye (Rotation) (Focus) RealWorld Parallax Over Pupil
  3. 3. Top View HMD Screen Stereo HMD Vergence & Accommodation Mismatch! Left Eye Right Eye RealWorld No Parallax! Parallax Over Pupil
  4. 4. Lenses Displays Drivers … … … $20 $70 $60 On ebay: DIY Light Field Stereoscope
  5. 5. Gaming
  6. 6. Da vinci surgical system
  7. 7. HEALTH and SAFETY WARNING
  8. 8. HEALTH and SAFETY WARNING
  9. 9. focus at the object’s distance
  10. 10. Vergence & accommodation are coupled!
  11. 11. Vergence & accommodation are coupled!
  12. 12. Immersive HMD Magnified Display
  13. 13. Immersive HMD Magnified Display
  14. 14. Decoupled! Immersive HMD Magnified Display !@#$$^R$%%&$ discomfort and fatigue, eyestrain, diplopic vision, headaches, nausea and …
  15. 15. 0.001 0.01 0.1 1 1m 10m 100m 1000m 10000m Personal Action Vista Relative Height Arial Perspective DepthContrast Distance [Cutting and Vishton 1995] Current HMD Depth Cues
  16. 16. Backlight Thin Spacer & 2nd panel (6mm) Magnifying Lenses LCD Panel
  17. 17. Front Focusing
  18. 18. Rear Focusing
  19. 19. Stereoscope Wheatstone, Brewster 1838 VR/AR Ivan Sutherland 1968 VR explosion Oculus, Sony, Valve, MS, … Now Future Future HMDs Resolution Latency FOV Form Factor … Accommodation
  20. 20. Stereoscope Wheatstone, Brewster 1838 VR/AR Ivan Sutherland 1968 VR explosion Oculus, Sony, Valve, MS, … Now Future Future HMDs [Lanman and Luebke 2013] [Hu and Hua 2014] Accommodation
  21. 21. Object emits natural light rays Light field enters eyes
  22. 22. Layered3D, Tensor Display [Wetzstein et al. 2011, 12] Small Angle Stationary
  23. 23. TV vs. HMD • Reduced brightness problematic • Shared: Need wide viewing angle – High Rank – Multiple Layers – Temporal Multiplexing • Viewer adapts to reduced brightness • Personal: Small viewing angle – Low Rank – 2 Layers – Native Refresh Rate
  24. 24. Low-rank, 2-panel No temporal multiplexing Both binocular & focus cues to form 3D perception
  25. 25. Low-rank, 2-panel No temporal multiplexing
  26. 26. Target Light Field Input: 4D light field for each eye
  27. 27. Multiplicative Two-layer Modulation 1 𝑙 𝑥, 𝑣 𝑥 𝑣 𝑡1 𝑡2 𝑙 𝑥, 𝑣 = 𝑡1 𝑥 𝑡2 𝑥 − (𝑣 − 𝑥) 𝑑 𝑒 𝑑 𝑒 Input: 4D light field for each eye
  28. 28. Multiplicative Two-layer Modulation 1 𝑡1 𝑡2 𝑙 𝑥, 𝑣1 = 𝑡1 𝑥 𝑡2 𝑥 − (𝑣1 − 𝑥) 𝑑 𝑒 𝑑 𝑒 Input: 4D light field for each eye
  29. 29. Multiplicative Two-layer Modulation 1 𝑡1 𝑡2 𝑙 𝑥, 𝑣4 = 𝑡1 𝑥 𝑡2 𝑥 − (𝑣4 − 𝑥) 𝑑 𝑒 𝑑 𝑒 Input: 4D light field for each eye
  30. 30. Multiplicative Two-layer Modulation 1 𝑡1 𝑡2 𝑙 𝑥, 𝑣7 = 𝑡1 𝑥 𝑡2 𝑥 − (𝑣7 − 𝑥) 𝑑 𝑒 𝑑 𝑒 Input: 4D light field for each eye
  31. 31. Multiplicative Two-layer Modulation 𝑡1 𝑡2 Input: 4D light field for each eye Parallax over the Pupil
  32. 32. Multiplicative Two-layer Modulation 𝑡1 𝑡2 l = (ϕ1 𝑡1)o(ϕ2 𝑡2)Reconstruction: Input: 4D light field for each eye 𝑚𝑖𝑛𝑖𝑚𝑖𝑧𝑒 𝛽l − ϕ1 𝑡1 o ϕ2 𝑡2 2 {𝑡1, 𝑡2} 𝑠. 𝑡. 0 ≤ 𝑡1, 𝑡2 ≤ 1 𝑡1 ← 𝑡1 o ϕ1 𝑇(𝛽l o (ϕ2 𝑡2)) ϕ1 𝑇 l o ϕ2 𝑡2 +𝜖 for layer t1 [Wetzstein et al 2012]
  33. 33. Input light field to the eyeMultiplicative Two-layer Modulation 𝑡1 𝑡2 Left Eye: Front Panel Left Eye: Rear Panel Front Panel Rear Panel Output: 2 layers for each eye
  34. 34. Front Focus Mid Focus Rear Focus Front Object Mid Object Rear Object Multiplicative Two-layer Modulation 𝑡1 𝑡2 Front Panel Rear Panel
  35. 35. Multilayer Displays Akeley et al. [2004] Love et al. [2009] Narain et al.[2015] Optical Overlay Temporal Multiplexing
  36. 36. Central Light Field
  37. 37. Right Light Field
  38. 38. Left Light Field With Additive Multilayer
  39. 39. Left Light Field Additive Zoom-In Multiplicative
  40. 40. The Hardware - Adafruit based 3D printed housing learn.adafruit.com/3d-printed-wearable -video-goggles - Aspheric lens, 2” diameter, 5cm focal length, $10 each - TOSHIBA LTD056EV7F, 5.6” 1280x800, $60 each - CHIMEI INNOLUX N070ICG-LD1, 7” $35 each (need to replace polarizer!) - PCB800099 driver board, $30 each
  41. 41. E-Tech Demo South Hall G, Booth ET18 The Hardware
  42. 42. Content Generation • 5x5x2 Stereo Light Field • GPU Factorization • 3 ~ 5 Iterations per frame • 25(views) x 640x800 x 2(eyes) • 5 ~ 10ms per iterations Left eye Right eye
  43. 43. Photograph Results Traditional HMDs - No Focus Cues The Light Field HMD Stereoscope
  44. 44. Photograph Results Traditional HMDs - No Focus Cues The Light Field HMD Stereoscope
  45. 45. Photograph Results Traditional HMDs - No Focus Cues The Light Field HMD Stereoscope
  46. 46. Photograph Results Traditional HMDs - No Focus Cues The Light Field HMD Stereoscope
  47. 47. Photograph Results Traditional HMDs - No Focus Cues The Light Field HMD Stereoscope
  48. 48. Photograph Results Traditional HMDs - No Focus Cues The Light Field HMD Stereoscope
  49. 49. Photograph Results Traditional HMDs - No Focus Cues The Light Field HMD Stereoscope
  50. 50. Photograph Results Traditional HMDs - No Focus Cues The Light Field HMD Stereoscope
  51. 51. Designing Real-world Experience Left eye Right eye Front Focusing Rear Focusing
  52. 52. Technical Details in the Paper Good Parameter Space Limited by Diffraction 100 200 300 400 500 2 3 5 6 4 Front Panel PPI NumberofViews Minimum Views to Support Accommodation IPD / 2 * M2 IPD / 2 * M1 Center Center Diffraction Analysis Light Field Distortion Asymmetric Image Formation
  53. 53. Discussion • Reduction in brightness • Latency: 5x5x2 views • Diffraction
  54. 54. Future Work Light Field Content Human Vision/Perception
  55. 55. IPD / 2 * M2 IPD / 2 * M1 Center Center Top View Light Field Side View Light Field

×