Downloaded 146 times
![From Conventional to Compressive 3D Displays
mask 2
mask 1
Time-shifted Parallax Barriers [Kim et al. 2007]
High Resolution through High Speed
t
t t
Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays
1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012](https://image.slidesharecdn.com/compressivelightfielddisplaysnoclips-121012172928-phpapp01/85/Compressive-Light-Field-Displays-22-320.jpg)
![From Conventional to Compressive 3D Displays
Perceptual Integration
time
Time-shifted Parallax Barriers [Kim et al. 2007]
High Resolution through High Speed
t
t t
Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays
1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012](https://image.slidesharecdn.com/compressivelightfielddisplaysnoclips-121012172928-phpapp01/85/Compressive-Light-Field-Displays-23-320.jpg)
![From Conventional to Compressive 3D Displays
mask 2
mask 1
High-Rank 3D [Lanman et al., SIGGRAPH Asia 2010]
Compression in Time – Nonnegative Matrix Factorization
t
t t
Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays
1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012](https://image.slidesharecdn.com/compressivelightfielddisplaysnoclips-121012172928-phpapp01/85/Compressive-Light-Field-Displays-24-320.jpg)
![From Conventional to Compressive 3D Displays
Perceptual Integration
time
High-Rank 3D [Lanman et al., SIGGRAPH Asia 2010]
Compression in Time – Nonnegative Matrix Factorization
t
t t
Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays
1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012](https://image.slidesharecdn.com/compressivelightfielddisplaysnoclips-121012172928-phpapp01/85/Compressive-Light-Field-Displays-25-320.jpg)
![From Conventional to Compressive 3D Displays
mask K
…
mask 2
mask 1
Layered 3D [Wetzstein et al., SIGGRAPH 2011]
Compression in Pixels & Depth – Computed Tomography
t
t t
Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays
1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012](https://image.slidesharecdn.com/compressivelightfielddisplaysnoclips-121012172928-phpapp01/85/Compressive-Light-Field-Displays-26-320.jpg)
![From Conventional to Compressive 3D Displays
mask K
…
mask 2
mask 1
Layered 3D [Wetzstein et al., SIGGRAPH 2011]
Compression in Pixels & Depth – Computed Tomography
t
t t
Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays
1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012](https://image.slidesharecdn.com/compressivelightfielddisplaysnoclips-121012172928-phpapp01/85/Compressive-Light-Field-Displays-27-320.jpg)
![From Conventional to Compressive 3D Displays
mask K
…
mask 2
mask 1
Layered 3D [Wetzstein et al., SIGGRAPH 2011]
Compression in Pixels – Computed Tomography
t
t t
Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays
1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012](https://image.slidesharecdn.com/compressivelightfielddisplaysnoclips-121012172928-phpapp01/85/Compressive-Light-Field-Displays-28-320.jpg)
Uploaded byGordon Wetzstein
Compressive Light Field Displays
This document summarizes research on compressive light field displays. It begins by introducing the concept of light field displays and their collaborators. It then provides examples of prototypes from layered 3D displays to tensor displays. Finally, it outlines the evolution of display technologies from conventional parallax barriers to the most recent compressive displays that use techniques like nonnegative matrix factorization and computed tomography to achieve compression in time, pixels, and depth for glasses-free 3D viewing.
More Related Content
Similar to Compressive Light Field Displays
Compressive Light Field Displays
- 1. Compressive This slide hasa 16:9 media window Light Field Displays Gordon Wetzstein - MIT Media Lab Collaborators: Doug Lanman, Matt Hirsch, Ramesh Raskar, Wolfgang Heidrich
- 2. This slide hasa 16:9 media window
- 4. viewer moves right Thisslide has a 16:9 media window viewer moves down 4D Light Field
- 5. Compressive Displays Computational Display Optics Processing
- 6. This slide hasa 16:9 media window Display-adaptive Compression Computed Tomography Nonnegative Tensor 4D Light Field Factorization Compressive Optics Uniform or Directional Backlight Stacked Layers (LCDs or Transparencies)
- 7. Prototype – Layered3D, SIGGRAPH 2011 Attenuation Layers with Spacers Backlight
- 8.
- 9. Prototype – TensorDisplay, SIGGRAPH 2012
- 10.
- 11. What do wemean by “glasses-free 3D”? binocular disparity convergence motion parallax accommodation/blur current glasses-based (stereoscopic) displays near-term glasses-free (light field) displays longer-term holographic displays
- 12. Is glasses-free 3DTechnology ready? Nintendo 3DS MasterImage 3D Asus Eee Pad MeMO 3D LG Optimus 3D E3 2010 Computex 2011 Computex 2011 Mobile World Congress 2011 Toshiba 3DTV Prototype Sony 3DTV Prototype LG 3DTV Prototype CES 2011 CES 2011 CES 2011
- 14. Parallax Barriers –Ives 1903 barrier 2D display Low resolution & very dim Switchable 2D/3D with LCDs
- 15. Parallax Barriers –Ives 1903 barrier Nintendo 3DS 2D display Low resolution & very dim Switchable 2D/3D with LCDs
- 16. lenslets Integral Imaging – Lippmann 1908 2D display Brighter than parallax barriers Always low resolution, even for 2D
- 17. lenslets Integral Imaging – Lippmann 1908 2D display Alioscopy 3DHD 42'' (1920x1200, 1x8 views) Brighter than parallax barriers Always low resolution, even for 2D
- 18. Directional Backlighting –3M & MS Wedge 3M Directional Backlight Film Nelson and Brott, 2010 US Patent 7,847,869 LED thin light guide LED Requires 120 Hz for stereo Not practical for multiview Microsoft Wedge
- 19. Directional Backlighting Lenslet array
- 20. Glasses-Free 3D Display LightSpace Sony Jones et al. 2007 Zebra Imaging MIT Holovideo Holograms Displays Volumetric 3Ddepth cues inside enclosure all objects only opticallymechanically moving parts mostly & computationally expensive 3D objects outside enclosure inexpensive off-the-shelf parts Compressive LF Displays no moving parts efficient computationally
- 21. From Conventional toCompressive 3D Displays mask 2 mask 1 Conventional Parallax Barriers t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 22. From Conventional toCompressive 3D Displays mask 2 mask 1 Time-shifted Parallax Barriers [Kim et al. 2007] High Resolution through High Speed t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 23. From Conventional toCompressive 3D Displays Perceptual Integration time Time-shifted Parallax Barriers [Kim et al. 2007] High Resolution through High Speed t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 24. From Conventional toCompressive 3D Displays mask 2 mask 1 High-Rank 3D [Lanman et al., SIGGRAPH Asia 2010] Compression in Time – Nonnegative Matrix Factorization t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 25. From Conventional toCompressive 3D Displays Perceptual Integration time High-Rank 3D [Lanman et al., SIGGRAPH Asia 2010] Compression in Time – Nonnegative Matrix Factorization t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 26. From Conventional toCompressive 3D Displays mask K … mask 2 mask 1 Layered 3D [Wetzstein et al., SIGGRAPH 2011] Compression in Pixels & Depth – Computed Tomography t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 27. From Conventional toCompressive 3D Displays mask K … mask 2 mask 1 Layered 3D [Wetzstein et al., SIGGRAPH 2011] Compression in Pixels & Depth – Computed Tomography t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 28. From Conventional toCompressive 3D Displays mask K … mask 2 mask 1 Layered 3D [Wetzstein et al., SIGGRAPH 2011] Compression in Pixels – Computed Tomography t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 29. From Conventional toCompressive 3D Displays Perceptual Integration … … … time Tensor Displays Compression in Time & Pixels –Tensor Factorization t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 30. From Conventional toCompressive 3D Displays Perceptual Integration … … … time Tensor Displays – Multilayer & Directional Backlighting t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 31. From Conventional toCompressive 3D Displays Perceptual Integration thin! time Tensor Displays – Directional Backlighting t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
- 33. Prototype – Layered3D, SIGGRAPH 2011 Attenuation Layers with Spacers Backlight
- 35. Computed Tomography (CT) x-ray sensor source: wikipedia 3D Reconstruction x-ray source Reconstructed 2D Slices 35
- 36. Tomographic Light FieldSynthesis Image Formation Virtual Planes - ò c m (r )dr x L(x, q ) = e Attenuation Volume log L x, (r )dr c Backlight Tomographic Synthesis 2D Light Field log( L ) P 2 argmin log( L) P 2 0 x 36
- 37. CT vs. Layered3D Computed Tomography Layered 3D reconstruct physical volume thin stack of optimized layers sensor noise no noise 37
- 38. Multi-Layer Decomposition viewer moves right viewer moves down Input 4D Light Field 1 2 3 1 2 4 3 5 4 5 Photographs of Prototype Optimized Attenuation Layers
- 39. Depth of Fieldfor 3D Displays Integral Imaging Parallax Barriers Cutoff (cycles/cm) Maximum Resolution Display Thickness Zwicker et al. 2006 Antialiasing + Display Prefilter Distance of Virtual Plane from Middle of Display (cm)
- 40. How Do LayersIncrease Depth of Field? Integral Imaging Parallax Barriers Layered 3D Cutoff (cycles/cm) ? Maximum Resolution Display Thickness Distance of Virtual Plane from Middle of Display (cm)
- 41. Optimization: Number ofLayers Two Layers Three Layers Five Layers
- 42. Optimization: Display Thickness Average Reconstruction PSNR for All Scenes PSNR in dB Number of Layers
- 43. Application to HDRDisplay “Square Root” Layers
- 44. Application to HDRDisplay “Square Root” Layers
- 45. Application to HDRDisplay Optimized Layers
- 46.
- 47. Limitations: Field ofView FOV 10º FOV 20º FOV 45º
- 48. Personal Glasses-Free 3DDisplay Challenges for dynamic display: Real-time computation Engineering issues, moiré
- 50. Multi-Layer LCD –SIGGRAPH ASIA 2011
- 51. Barco E-2320 PA Grayscale IPS LCD 1600x1200 @ 60 Hz
- 52. Four Stacked LiquidCrystal Panels Two Crossed Polarizers
- 53. Overview of LCDs I vertical polarizer color filter array liquid crystal cells horizontal polarizer I0 backlight Malus’ Law Intensity Modulation with Liquid Crystal Cells I = I0 sin2 (q )
- 54. Extending Layered 3Dto Multi-layer LCDs Virtual Planes Design Optimization LCD • Eliminate redundant polarizers 3 Sequentially-crossed design LCD x 2 LCD 1 backlight 2D Light Field x
- 55. Extending Layered 3Dto Multi-layer LCDs Virtual Planes Design Optimization LCD • Eliminate redundant polarizers 3 Use sequentially-crossed LCD x • Exploit field-sequential color 2 0.33 = 2.7% brightness LCD 1 backlight 2D Light Field x
- 56. Polarization Field Displays Virtual Planes Design Optimization LCD • Eliminate redundant polarizers 3 Use sequentially-crossed LCD x • Exploit field-sequential color 2 0.33 = 2.7% brightness LCD • Further optimize polarizers 1 Minimum is a crossed pair backlight 2D Light Field x
- 57. Modeling and SynthesizingPolarization Fields Virtual Planes LCD f3 3 Image Formation f2 K LCD x Q(x, q ) = åfk (x, q ) 2 k=1 f1 L(x, q ) = sin 2 (Q(x,q )) LCD 1 backlight Tomographic Synthesis Q(x,q ) = ±sin-1 ( ) L(x, q ) mod p 2D Light Field Q = Pf argmin Q - Pf 2 2 fmin £f £fmax x
- 58. Decompositions & Reconstructions viewer moves right viewer moves down Stacked Polarization Input 4D Light Field Rotating Layers 90° Optimized Rotation Angles for Each Layer 0°
- 59. Decompositions & Reconstructions viewer moves right viewer moves down Input 4D Light Field Reconstruction Results
- 60. Multi-layer LCD Attenuation Layers Polarization-Rotating Layers
- 61.
- 62. Tomographic Image Synthesis Target Light Field Projection Matrix LCD Pixel Values = * b=Ax
- 63. SART - Simultaneous Algebraic Reconstruction Technique b=Ax pre-compute some weights initial guess ATv Ax update clamp
- 64. Implementing Ax asMultiview Rendering
- 65. Implementing ATv asProjective Texture Mapping
- 66.
- 68. Light Field “Slice”Representation Light Field moving to the right
- 69. Light Field “Slice”Representation Light Field Light Field Slice moving to the left
- 70. Light Field “Slice”Representation Light Field View from Above Light Field Slice moving to the left
- 71. Light Field “Slice”Representation Light Field Multilayer Light Field Display Light Field Slice moving to the left Front Layer Middle Layer Rear Layer Backlight
- 72. Light Field “Slice”Representation Light Field Multilayer Light Field Display Light Field Slice L( moving to the left Front Layer fm(3)( Middle Layer fm(2)( L( Rear Layer fm(1)( Backlight
- 73. Light Field TensorRepresentation Light Field Multilayer Light Field Display Light Field Tensor L( Front Layer Rear Layer fm(3)( Middle Layer L( fm(2)( Rear Layer fm(1)( Backlight
- 74. Light Field TensorRepresentation Light Field Multilayer Light Field Display Light Field Tensor L( Front Layer Rear Layer fm(3)( Middle Layer L( fm(2)( Rear Layer fm(1)( Backlight
- 75. Light Field TensorRepresentation Light Field Multilayer Light Field Display Light Field Tensor Front Layer Rear Layer fm(3)( Middle Layer fm(2)( Rear Layer fm(1)( Backlight
- 76. Light Field TensorRepresentation Light Field Multilayer Light Field Display Light Field Tensor Front Layer Rear Layer fm(3)( Middle Layer fm(2)( Rear Layer fm(1)( Backlight
- 77. Light Field TensorRepresentation Light Field Multilayer Light Field Display Light Field Tensor Front Layer Rear Layer fm(3)( Middle Layer fm(2)( Rear Layer fm(1)( Backlight
- 78. Light Field TensorDecomposition Target Light Field Tensor Rank-M Approximation Nonnegative Tensor Perceptual Factorization (NTF) Integration + + ... + Frame 1 Frame 2 Frame M
- 79. Light Field TensorDecomposition Target Light Field Tensor Rank-M Approximation Nonnegative Tensor Perceptual Factorization (NTF) Integration + + ... + Frame 1 Frame 2 Frame M
- 80. Light Field TensorDecomposition Nonlinear (Multilinear) Optimization Problem Iterative Update Rules (see paper for details) Efficient GPU Implementation Forward Projection (Multiview Rendering) Back Projection (Projective Texture Mapping)
- 82. Design Tradespace: Layersvs. Frames PSNR without Directional Backlight # layers # frames PSNR with Directional Backlight # layers # frames
- 83. Design Tradespace: Layersvs. Frames PSNR without Directional Backlight # layers # frames PSNR with Directional Backlight # layers # frames 2 Layers, Layers,Directional–Backlight (Tensor Display) 3 Layers, 1 Frame Frames 3D – SIGGRAPH 2010) 1 Layer, 3 3 3 Frames(Layered (Tensor Display) 2011) Frames, 3 (HR3D SIGGRAPH Asia Original Original 2 Layers, 3 Frames 3 Layers, 1 Frame 3 Layers, 3 Frames 1 L, 3 F, Directional BL
- 84. Tensor Display Prototypes Reconfigurable Directional Backlight Three Layer
- 85. Tensor Display Prototypes 3 Layer LCD Directional Backlight Hardware
- 86. Three Layer Prototype Video clip
- 87.
- 88. Directional Backlight Prototype Video clip
- 89. LCD + DirectionalBL View from above LCD with Directional Backlight, Rank 6 Directional BL
- 90. LCD with DirectionalBacklight, Rank 6 (as seen by obserer) Video clip
- 91. LCD with DirectionalBacklight, Rank 6 (as seen by obserer) Video clip
- 92. Filmed with High-speedCamera Directional Backlight Video clip Front LCD
- 93. Video clip Lenslets only LCD+DBL Rank 1 LCD+DBL Rank 6
- 94. Limitation: Compressible LightFields “Natural” 4D Light Field Random 4D Light Field
- 95. Next-generation Technology What about Content? Computational Photography Consumer Light Field Cameras Rendered Footage Computational Displays Camera Rigs
- 96. SIGGRAPH 2012 Courseon Computational Displays Code & Datasets online Use Layered 3D in your class! media.mit.edu/~gordonw cameraculture.media.mit.edu
Editor's Notes
- #48 [Gordon resumes at this slide.]
- #49 Multi-layer 3D display is a computed tomography problem. (Multi-Layer is 3D)Natural light fields are compressible, so a thin display with few layers can achieve the illusion of a thick virtual scene. (thin display creates thick scenes)So this leads to a new type of computational/compressive display.[probably same slide as personal device]
- #63 Basically, the tomographic light field synthesis then boils down to solving a linear equation system of the form Ax=b. b is the target light field, A the projection matrix, and the unknowns x are the LCD pixel values
- #64 This equation system can be solved with SART – a technique developed, tested, and refined over decades in the medical imaging community.Algorithm is very simple:Initialize some data and find an initial guess of the pixel valuesIteratively update and clamp the solutionUpdate rules require two important operations: a function computing the matrix-vector multiplication Ax, and a function computing the transpose matrix-vector multiplication ATv
- #65 Interpreting AX – is basically a multiview rendering step. In OpenGL just set the projection matrix to all different views of the light field and render the LCD layers textured with the given patterns.
- #66 Interpreting AX – is basically a multiview rendering step. In OpenGL just set the projection matrix to all different views of the light field and render the LCD layers textured with the given patterns.
- #90 Use the original video clips in higher res and the same layout but better quality and not this!Get rid of the gray background as well
- #91 Use the original video clips in higher res and the same layout but better quality and not this!Get rid of the gray background as well
- #92 Use the original video clips in higher res and the same layout but better quality and not this!Get rid of the gray background as well
- #94 Use the original video clips in higher res and the same layout but better quality and not this!Get rid of the gray background as wellPossibly use static framesEmphasize