Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

of

Compressive Light Field Displays Slide 1 Compressive Light Field Displays Slide 2 Compressive Light Field Displays Slide 3 Compressive Light Field Displays Slide 4 Compressive Light Field Displays Slide 5 Compressive Light Field Displays Slide 6 Compressive Light Field Displays Slide 7 Compressive Light Field Displays Slide 8 Compressive Light Field Displays Slide 9 Compressive Light Field Displays Slide 10 Compressive Light Field Displays Slide 11 Compressive Light Field Displays Slide 12 Compressive Light Field Displays Slide 13 Compressive Light Field Displays Slide 14 Compressive Light Field Displays Slide 15 Compressive Light Field Displays Slide 16 Compressive Light Field Displays Slide 17 Compressive Light Field Displays Slide 18 Compressive Light Field Displays Slide 19 Compressive Light Field Displays Slide 20 Compressive Light Field Displays Slide 21 Compressive Light Field Displays Slide 22 Compressive Light Field Displays Slide 23 Compressive Light Field Displays Slide 24 Compressive Light Field Displays Slide 25 Compressive Light Field Displays Slide 26 Compressive Light Field Displays Slide 27 Compressive Light Field Displays Slide 28 Compressive Light Field Displays Slide 29 Compressive Light Field Displays Slide 30 Compressive Light Field Displays Slide 31 Compressive Light Field Displays Slide 32 Compressive Light Field Displays Slide 33 Compressive Light Field Displays Slide 34 Compressive Light Field Displays Slide 35 Compressive Light Field Displays Slide 36 Compressive Light Field Displays Slide 37 Compressive Light Field Displays Slide 38 Compressive Light Field Displays Slide 39 Compressive Light Field Displays Slide 40 Compressive Light Field Displays Slide 41 Compressive Light Field Displays Slide 42 Compressive Light Field Displays Slide 43 Compressive Light Field Displays Slide 44 Compressive Light Field Displays Slide 45 Compressive Light Field Displays Slide 46 Compressive Light Field Displays Slide 47 Compressive Light Field Displays Slide 48 Compressive Light Field Displays Slide 49 Compressive Light Field Displays Slide 50 Compressive Light Field Displays Slide 51 Compressive Light Field Displays Slide 52 Compressive Light Field Displays Slide 53 Compressive Light Field Displays Slide 54 Compressive Light Field Displays Slide 55 Compressive Light Field Displays Slide 56 Compressive Light Field Displays Slide 57 Compressive Light Field Displays Slide 58 Compressive Light Field Displays Slide 59 Compressive Light Field Displays Slide 60 Compressive Light Field Displays Slide 61 Compressive Light Field Displays Slide 62 Compressive Light Field Displays Slide 63 Compressive Light Field Displays Slide 64 Compressive Light Field Displays Slide 65 Compressive Light Field Displays Slide 66 Compressive Light Field Displays Slide 67 Compressive Light Field Displays Slide 68 Compressive Light Field Displays Slide 69 Compressive Light Field Displays Slide 70 Compressive Light Field Displays Slide 71 Compressive Light Field Displays Slide 72 Compressive Light Field Displays Slide 73 Compressive Light Field Displays Slide 74 Compressive Light Field Displays Slide 75 Compressive Light Field Displays Slide 76 Compressive Light Field Displays Slide 77 Compressive Light Field Displays Slide 78 Compressive Light Field Displays Slide 79 Compressive Light Field Displays Slide 80 Compressive Light Field Displays Slide 81 Compressive Light Field Displays Slide 82 Compressive Light Field Displays Slide 83 Compressive Light Field Displays Slide 84 Compressive Light Field Displays Slide 85 Compressive Light Field Displays Slide 86 Compressive Light Field Displays Slide 87 Compressive Light Field Displays Slide 88 Compressive Light Field Displays Slide 89 Compressive Light Field Displays Slide 90 Compressive Light Field Displays Slide 91 Compressive Light Field Displays Slide 92 Compressive Light Field Displays Slide 93 Compressive Light Field Displays Slide 94 Compressive Light Field Displays Slide 95 Compressive Light Field Displays Slide 96
Upcoming SlideShare
Compressive DIsplays: SID Keynote by Ramesh Raskar
Next
Download to read offline and view in fullscreen.

6 Likes

Share

Download to read offline

Compressive Light Field Displays

Download to read offline

Overview of a new generation of glasses-free 3D (light field) displays.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Compressive Light Field Displays

  1. 1. Compressive This slide has a 16:9 media window Light Field Displays Gordon Wetzstein - MIT Media Lab Collaborators: Doug Lanman, Matt Hirsch, Ramesh Raskar, Wolfgang Heidrich
  2. 2. This slide has a 16:9 media window
  3. 3. viewer moves right This slide has a 16:9 media window viewer moves down 4D Light Field
  4. 4. Compressive Displays Computational Display Optics Processing
  5. 5. This slide has a 16:9 media window Display-adaptive Compression Computed Tomography Nonnegative Tensor 4D Light Field Factorization Compressive Optics Uniform or Directional Backlight Stacked Layers (LCDs or Transparencies)
  6. 6. Prototype – Layered 3D, SIGGRAPH 2011 Attenuation Layers with Spacers Backlight
  7. 7. Video clip
  8. 8. Prototype – Tensor Display, SIGGRAPH 2012
  9. 9. Video clip
  10. 10. What do we mean by “glasses-free 3D”? binocular disparity convergence motion parallax accommodation/blur current glasses-based (stereoscopic) displays near-term glasses-free (light field) displays longer-term holographic displays
  11. 11. Is glasses-free 3D Technology ready? Nintendo 3DS MasterImage 3D Asus Eee Pad MeMO 3D LG Optimus 3D E3 2010 Computex 2011 Computex 2011 Mobile World Congress 2011 Toshiba 3DTV Prototype Sony 3DTV Prototype LG 3DTV Prototype CES 2011 CES 2011 CES 2011
  12. 12. Parallax Barriers – Ives 1903 barrier 2D display  Low resolution & very dim  Switchable 2D/3D with LCDs
  13. 13. Parallax Barriers – Ives 1903 barrier Nintendo 3DS 2D display  Low resolution & very dim  Switchable 2D/3D with LCDs
  14. 14. lenslets Integral Imaging – Lippmann 1908 2D display  Brighter than parallax barriers  Always low resolution, even for 2D
  15. 15. lenslets Integral Imaging – Lippmann 1908 2D display Alioscopy 3DHD 42'' (1920x1200, 1x8 views)  Brighter than parallax barriers  Always low resolution, even for 2D
  16. 16. Directional Backlighting – 3M & MS Wedge 3M Directional Backlight Film Nelson and Brott, 2010 US Patent 7,847,869 LED thin light guide LED  Requires 120 Hz for stereo  Not practical for multiview Microsoft Wedge
  17. 17. Directional Backlighting Lenslet array
  18. 18. Glasses-Free 3D Display LightSpace Sony Jones et al. 2007 Zebra Imaging MIT Holovideo Holograms Displays Volumetric  3Ddepth cues inside enclosure all objects only  opticallymechanically moving parts  mostly & computationally expensive  3D objects outside enclosure inexpensive off-the-shelf parts Compressive LF Displays  no moving parts efficient computationally
  19. 19. From Conventional to Compressive 3D Displays mask 2 mask 1 Conventional Parallax Barriers t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
  20. 20. From Conventional to Compressive 3D Displays mask 2 mask 1 Time-shifted Parallax Barriers [Kim et al. 2007] High Resolution through High Speed t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
  21. 21. From Conventional to Compressive 3D Displays Perceptual Integration time Time-shifted Parallax Barriers [Kim et al. 2007] High Resolution through High Speed t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
  22. 22. From Conventional to Compressive 3D Displays mask 2 mask 1 High-Rank 3D [Lanman et al., SIGGRAPH Asia 2010] Compression in Time – Nonnegative Matrix Factorization t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
  23. 23. From Conventional to Compressive 3D Displays Perceptual Integration time High-Rank 3D [Lanman et al., SIGGRAPH Asia 2010] Compression in Time – Nonnegative Matrix Factorization t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
  24. 24. From Conventional to Compressive 3D Displays mask K … mask 2 mask 1 Layered 3D [Wetzstein et al., SIGGRAPH 2011] Compression in Pixels & Depth – Computed Tomography t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
  25. 25. From Conventional to Compressive 3D Displays mask K … mask 2 mask 1 Layered 3D [Wetzstein et al., SIGGRAPH 2011] Compression in Pixels & Depth – Computed Tomography t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
  26. 26. From Conventional to Compressive 3D Displays mask K … mask 2 mask 1 Layered 3D [Wetzstein et al., SIGGRAPH 2011] Compression in Pixels – Computed Tomography t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
  27. 27. From Conventional to Compressive 3D Displays Perceptual Integration … … … time Tensor Displays Compression in Time & Pixels –Tensor Factorization t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
  28. 28. From Conventional to Compressive 3D Displays Perceptual Integration … … … time Tensor Displays – Multilayer & Directional Backlighting t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
  29. 29. From Conventional to Compressive 3D Displays Perceptual Integration thin! time Tensor Displays – Directional Backlighting t t t Parallax Barriers Time-Shifted HR3D Layered 3D Tensor Displays 1903 Parallax Barriers 2007 SIG Asia 2010 SIGGRAPH 2011 SIGGRAPH 2012
  30. 30. Prototype – Layered 3D, SIGGRAPH 2011 Attenuation Layers with Spacers Backlight
  31. 31. Computed Tomography (CT) x-ray sensor source: wikipedia 3D Reconstruction x-ray source Reconstructed 2D Slices 35
  32. 32. Tomographic Light Field Synthesis Image Formation Virtual Planes - ò c m (r )dr x L(x, q ) = e Attenuation Volume log L x, (r )dr c Backlight Tomographic Synthesis 2D Light Field log( L ) P 2 argmin log( L) P 2 0 x 36
  33. 33. CT vs. Layered 3D Computed Tomography Layered 3D  reconstruct physical volume  thin stack of optimized layers  sensor noise  no noise 37
  34. 34. Multi-Layer Decomposition viewer moves right viewer moves down Input 4D Light Field 1 2 3 1 2 4 3 5 4 5 Photographs of Prototype Optimized Attenuation Layers
  35. 35. Depth of Field for 3D Displays Integral Imaging Parallax Barriers Cutoff (cycles/cm) Maximum Resolution Display Thickness Zwicker et al. 2006 Antialiasing + Display Prefilter Distance of Virtual Plane from Middle of Display (cm)
  36. 36. How Do Layers Increase Depth of Field? Integral Imaging Parallax Barriers Layered 3D Cutoff (cycles/cm) ? Maximum Resolution Display Thickness Distance of Virtual Plane from Middle of Display (cm)
  37. 37. Optimization: Number of Layers Two Layers Three Layers Five Layers
  38. 38. Optimization: Display Thickness Average Reconstruction PSNR for All Scenes PSNR in dB Number of Layers
  39. 39. Application to HDR Display “Square Root” Layers
  40. 40. Application to HDR Display “Square Root” Layers
  41. 41. Application to HDR Display Optimized Layers
  42. 42. Video clip
  43. 43. Limitations: Field of View FOV 10º FOV 20º FOV 45º
  44. 44. Personal Glasses-Free 3D Display Challenges for dynamic display:  Real-time computation  Engineering issues, moiré
  45. 45. Multi-Layer LCD – SIGGRAPH ASIA 2011
  46. 46. Barco E-2320 PA Grayscale IPS LCD 1600x1200 @ 60 Hz
  47. 47. Four Stacked Liquid Crystal Panels Two Crossed Polarizers
  48. 48. Overview of LCDs I vertical polarizer color filter array liquid crystal cells horizontal polarizer I0 backlight Malus’ Law Intensity Modulation with Liquid Crystal Cells I = I0 sin2 (q )
  49. 49. Extending Layered 3D to Multi-layer LCDs Virtual Planes Design Optimization LCD • Eliminate redundant polarizers 3  Sequentially-crossed design LCD x 2 LCD 1 backlight 2D Light Field x
  50. 50. Extending Layered 3D to Multi-layer LCDs Virtual Planes Design Optimization LCD • Eliminate redundant polarizers 3  Use sequentially-crossed LCD x • Exploit field-sequential color 2  0.33 = 2.7% brightness LCD 1 backlight 2D Light Field x
  51. 51. Polarization Field Displays Virtual Planes Design Optimization LCD • Eliminate redundant polarizers 3  Use sequentially-crossed LCD x • Exploit field-sequential color 2  0.33 = 2.7% brightness LCD • Further optimize polarizers 1  Minimum is a crossed pair backlight 2D Light Field x
  52. 52. Modeling and Synthesizing Polarization Fields Virtual Planes LCD f3 3 Image Formation f2 K LCD x Q(x, q ) = åfk (x, q ) 2 k=1 f1 L(x, q ) = sin 2 (Q(x,q )) LCD 1 backlight Tomographic Synthesis Q(x,q ) = ±sin-1 ( ) L(x, q ) mod p 2D Light Field Q = Pf argmin Q - Pf 2 2 fmin £f £fmax x
  53. 53. Decompositions & Reconstructions viewer moves right viewer moves down Stacked Polarization Input 4D Light Field Rotating Layers 90° Optimized Rotation Angles for Each Layer 0°
  54. 54. Decompositions & Reconstructions viewer moves right viewer moves down Input 4D Light Field Reconstruction Results
  55. 55. Multi-layer LCD Attenuation Layers Polarization-Rotating Layers
  56. 56. Video clip
  57. 57. Tomographic Image Synthesis Target Light Field Projection Matrix LCD Pixel Values = * b=Ax
  58. 58. SART - Simultaneous Algebraic Reconstruction Technique b=Ax pre-compute some weights initial guess ATv Ax update clamp
  59. 59. Implementing Ax as Multiview Rendering
  60. 60. Implementing ATv as Projective Texture Mapping
  61. 61. Benefits & Limitations
  62. 62. Light Field “Slice” Representation Light Field moving to the right
  63. 63. Light Field “Slice” Representation Light Field Light Field Slice moving to the left
  64. 64. Light Field “Slice” Representation Light Field View from Above Light Field Slice moving to the left
  65. 65. Light Field “Slice” Representation Light Field Multilayer Light Field Display Light Field Slice moving to the left Front Layer Middle Layer Rear Layer Backlight
  66. 66. Light Field “Slice” Representation Light Field Multilayer Light Field Display Light Field Slice L( moving to the left Front Layer fm(3)( Middle Layer fm(2)( L( Rear Layer fm(1)( Backlight
  67. 67. Light Field Tensor Representation Light Field Multilayer Light Field Display Light Field Tensor L( Front Layer Rear Layer fm(3)( Middle Layer L( fm(2)( Rear Layer fm(1)( Backlight
  68. 68. Light Field Tensor Representation Light Field Multilayer Light Field Display Light Field Tensor L( Front Layer Rear Layer fm(3)( Middle Layer L( fm(2)( Rear Layer fm(1)( Backlight
  69. 69. Light Field Tensor Representation Light Field Multilayer Light Field Display Light Field Tensor Front Layer Rear Layer fm(3)( Middle Layer fm(2)( Rear Layer fm(1)( Backlight
  70. 70. Light Field Tensor Representation Light Field Multilayer Light Field Display Light Field Tensor Front Layer Rear Layer fm(3)( Middle Layer fm(2)( Rear Layer fm(1)( Backlight
  71. 71. Light Field Tensor Representation Light Field Multilayer Light Field Display Light Field Tensor Front Layer Rear Layer fm(3)( Middle Layer fm(2)( Rear Layer fm(1)( Backlight
  72. 72. Light Field Tensor Decomposition Target Light Field Tensor Rank-M Approximation Nonnegative Tensor Perceptual Factorization (NTF) Integration + + ... + Frame 1 Frame 2 Frame M
  73. 73. Light Field Tensor Decomposition Target Light Field Tensor Rank-M Approximation Nonnegative Tensor Perceptual Factorization (NTF) Integration + + ... + Frame 1 Frame 2 Frame M
  74. 74. Light Field Tensor Decomposition Nonlinear (Multilinear) Optimization Problem Iterative Update Rules (see paper for details) Efficient GPU Implementation Forward Projection (Multiview Rendering) Back Projection (Projective Texture Mapping)
  75. 75. Design Tradespace: Layers vs. Frames PSNR without Directional Backlight # layers # frames PSNR with Directional Backlight # layers # frames
  76. 76. Design Tradespace: Layers vs. Frames PSNR without Directional Backlight # layers # frames PSNR with Directional Backlight # layers # frames 2 Layers, Layers,Directional–Backlight (Tensor Display) 3 Layers, 1 Frame Frames 3D – SIGGRAPH 2010) 1 Layer, 3 3 3 Frames(Layered (Tensor Display) 2011) Frames, 3 (HR3D SIGGRAPH Asia Original Original 2 Layers, 3 Frames 3 Layers, 1 Frame 3 Layers, 3 Frames 1 L, 3 F, Directional BL
  77. 77. Tensor Display Prototypes Reconfigurable Directional Backlight Three Layer
  78. 78. Tensor Display Prototypes 3 Layer LCD Directional Backlight Hardware
  79. 79. Three Layer Prototype Video clip
  80. 80. Video clip
  81. 81. Directional Backlight Prototype Video clip
  82. 82. LCD + Directional BL View from above LCD with Directional Backlight, Rank 6 Directional BL
  83. 83. LCD with Directional Backlight, Rank 6 (as seen by obserer) Video clip
  84. 84. LCD with Directional Backlight, Rank 6 (as seen by obserer) Video clip
  85. 85. Filmed with High-speed Camera Directional Backlight Video clip Front LCD
  86. 86. Video clip Lenslets only LCD+DBL Rank 1 LCD+DBL Rank 6
  87. 87. Limitation: Compressible Light Fields “Natural” 4D Light Field Random 4D Light Field
  88. 88. Next-generation Technology What about Content? Computational Photography Consumer Light Field Cameras Rendered Footage Computational Displays Camera Rigs
  89. 89. SIGGRAPH 2012 Course on Computational Displays Code & Datasets online Use Layered 3D in your class! media.mit.edu/~gordonw cameraculture.media.mit.edu
  • WonheeChoe

    Aug. 22, 2018
  • kaootao

    Mar. 4, 2018
  • huahelen1

    Aug. 3, 2015
  • ivanlin393

    Dec. 22, 2014
  • JianJia

    Apr. 22, 2014
  • w_keikou_w

    Aug. 25, 2013

Overview of a new generation of glasses-free 3D (light field) displays.

Views

Total views

5,449

On Slideshare

0

From embeds

0

Number of embeds

845

Actions

Downloads

146

Shares

0

Comments

0

Likes

6

×