SIGGRAPH 2012 Computational Display Course - 3 Computational Light Field Displays

7,753 views

Published on

Published in: Technology, Business
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
7,753
On SlideShare
0
From Embeds
0
Number of Embeds
4,583
Actions
Shares
0
Downloads
154
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide
  • Basically, the tomographic light field synthesis then boils down to solving a linear equation system of the form Ax=b. b is the target light field, A the projection matrix, and the unknowns x are the LCD pixel values
  • This equation system can be solved with SART – a technique developed, tested, and refined over decades in the medical imaging community.Algorithm is very simple:Initialize some data and find an initial guess of the pixel valuesIteratively update and clamp the solutionUpdate rules require two important operations: a function computing the matrix-vector multiplication Ax, and a function computing the transpose matrix-vector multiplication ATv
  • SIGGRAPH 2012 Computational Display Course - 3 Computational Light Field Displays

    1. 1. Edit this text to create aa 16:9 media This slide has Heading Computational Light Field window Displays This subtitle is 20 points Bullets are blue They have 110% line spacing, 2 points before & after Longer bullets in the form of a paragraph are harder to read if there is insufficient line spacing. This is the maximum recommended number of lines per slide (seven). Douglas Lanman Matthew Hirsch  Sub bulletsMIT Media Lab look like this MIT Media Lab
    2. 2. Is “glasses-free 3D” ready? Nintendo 3DS MasterImage 3D Asus Eee Pad MeMO 3D LG Optimus 3D E3 2010 Computex 2011 Computex 2011 Mobile World Congress 2011 Toshiba 3DTV Prototype Sony 3DTV Prototype LG 3DTV Prototype CES 2011 CES 2011 CES 2011
    3. 3. Taxonomy of Direct 3D Displays: Glasses-bound vs. Unencumbered Designs Immersive (blocks direct-viewing of real world) Head-mounted (eyepiece-objective and microdisplay) See-through (superimposes synthetic images onto real world) Glasses-bound Stereoscopic Spatially-multiplexed (field-concurrent) (color filters, polarizers, autostereograms, etc.) Multiplexed (stereo pair with same display surface) Temporally-multiplexed (field-sequential) (LCD shutter glasses) Parallax Barriers (uniform array of 1D slits or 2D pinhole arrays) Parallax-based (2D display with light-directing elements) Integral Imaging (lenticular sheets or fly’s eye lenslet arrays) Multi-planar Unencumbered (time-sequential projection onto swept surfaces) Volumetric (directly illuminate points within a volume) Transparent Substrates Automultiscopic (intersecting laser beams, fog layers, etc.) Static (holographic films) Holographic (reconstructs wavefront using 2D element) Dynamic (holovideo)Taxonomy adapted from Hong Hua
    4. 4. Taxonomy of Direct 3D Displays: Parallax Barriers NewSight MV-42AD3 42 (1920x1080, 1x8 views) Parallax Barriers (uniform array of 1D slits or 2D pinhole arrays) Parallax-based (2D display with light-directing elements)Unencumbered Volumetric (directly illuminate points within a volume)Automultiscopic Holographic (reconstructs wavefront using 2D element)
    5. 5. Taxonomy of Direct 3D Displays: Integral Imaging Alioscopy 3DHD 42 (1920x1200, 1x8 views) Parallax Barriers (uniform array of 1D slits or 2D pinhole arrays) Parallax-based (2D display with light-directing elements) Integral Imaging (lenticular sheets or fly’s eye lenslet arrays)Unencumbered Volumetric (directly illuminate points within a volume)Automultiscopic Holographic (reconstructs wavefront using 2D element)
    6. 6. Directional Backlighting Nelson and Brott, 2010 US Patent 7,847,869 Currently promoted by 3M Requires a high-speed (120 Hz) LCD panel, an additional double-sided prism film, and a pair of LEDs Allows multi-view display, but requires higher-speed LCD and additional light sources for each view
    7. 7. Taxonomy of Direct 3D Displays: Multi-planar Volumetric Displays Parallax Barriers (uniform array of 1D slits or 2D pinhole arrays) Parallax-based (2D display with light-directing elements) Integral Imaging (lenticular sheets or fly’s eye lenslet arrays) Multi-planarUnencumbered (time-sequential projection onto swept surfaces) Volumetric (directly illuminate points within a volume)Automultiscopic Holographic (reconstructs wavefront using 2D element)
    8. 8. Taxonomy of Direct 3D Displays: Transparent-substrate Volumetric Displays Parallax Barriers (uniform array of 1D slits or 2D pinhole arrays) Parallax-based (2D display with light-directing elements) Integral Imaging (lenticular sheets or fly’s eye lenslet arrays) Multi-planarUnencumbered (time-sequential projection onto swept surfaces) Volumetric (directly illuminate points within a volume) Transparent SubstratesAutomultiscopic (intersecting laser beams, fog layers, etc.) Holographic (reconstructs wavefront using 2D element)
    9. 9. Taxonomy of Direct 3D Displays: Static Holograms capture reconstruction Parallax Barriers (uniform array of 1D slits or 2D pinhole arrays) Parallax-based (2D display with light-directing elements) Integral Imaging (lenticular sheets or fly’s eye lenslet arrays) Multi-planarUnencumbered (time-sequential projection onto swept surfaces) Volumetric (directly illuminate points within a volume) Transparent SubstratesAutomultiscopic (intersecting laser beams, fog layers, etc.) Static (holographic films) Holographic (reconstructs wavefront using 2D element)
    10. 10. Taxonomy of Direct 3D Displays: Dynamic Holograms (Holovideo) Tay et al. MIT Media Lab Spatial Imaging Group [Nature, 2008] [Holovideo, 1989 – present] Parallax Barriers (uniform array of 1D slits or 2D pinhole arrays) Parallax-based (2D display with light-directing elements) Integral Imaging (lenticular sheets or fly’s eye lenslet arrays) Multi-planarUnencumbered (time-sequential projection onto swept surfaces) Volumetric (directly illuminate points within a volume) Transparent SubstratesAutomultiscopic (intersecting laser beams, fog layers, etc.) Static (holographic films) Holographic (reconstructs wavefront using 2D element) Dynamic (holovideo)
    11. 11. What is meant by “glasses-free 3D”?binocular disparity convergence motion parallax accommodation/blurcurrent glasses-based (stereoscopic) displaysnear-term glasses-free (automultiscopic) displayslonger-term volumetric and holographic displays
    12. 12. Design Trade-offs Integral Imaging Parallax Barriers Directional Backlighting Integral Imaging Parallax Barriers Directional Backlighting Spatial Resolution low low high Brightness high low moderate Cost low low – moderate moderate – high Full-resolution 2D no yes (dual-layer LCD) yes Motion Parallax yes yes no
    13. 13. Generalizing Parallax Barriers mask K … mask 3 mask 2 mask 2 mask 2 mask 1 mask 1 mask 1 light box light box light box Conventional Parallax Barrier High-Rank 3D (HR3D) Layered 3D and Polarization Fields Parallax barriers use heuristic design: front mask with slits/pinholes, rear mask with interlaced views High-Rank 3D (HR3D) considers dual-layer design with arbitrary opacity and temporal multiplexing Layered 3D and Polarization Fields considers multi-layer design without temporal multiplexing
    14. 14. Outline Automultiscopic Displays  Multi-Layer Displays  Layered 3D - Polarization Fields  Dual-Layer Displays - High-Rank 3D (HR3D)
    15. 15. Layered 3D: Multi-Layer Displays mask K … mask 3 mask 2 mask 1 light box Layered 3D
    16. 16. Tomographic Light Field Synthesis virtual plane q attenuator x backlight q x 2D Light Field
    17. 17. Tomographic Light Field Synthesis virtual plane attenuator x backlight q x 2D Light Field
    18. 18. Tomographic Light Field Synthesis virtual plane attenuator x backlight q x 2D Light Field
    19. 19. Tomographic Light Field Synthesis virtual plane Image formation model: ò - m (r )dr attenuator L(x, q ) = I 0 e C æ L(x, q ) ö L(x, q ) = ln ç ÷ = - ò m (r)dr è I0 ø C backlight l = -Pa Tomographic synthesis: 2 arg min l + Pa , for a ³ 0 a 2D Light Field
    20. 20. Tomographic Light Field Synthesis virtual plane Image formation model: ò - m (r )dr attenuator L(x, q ) = I 0 e C æ L(x, q ) ö L(x, q ) = ln ç ÷ = - ò m (r)dr è I0 ø C backlight l = -Pa Tomographic synthesis: 2 arg min l + Pa , for a ³ 0 a 2D Light Field
    21. 21. Multi-Layer Light Field Decomposition Reconstructed Views Target 4D Light Field Multi-Layer Decomposition
    22. 22. Prototype Layered 3D Display Transparency stack with acrylic spacers Prototype in front of LCD (backlight source)
    23. 23. Outline Automultiscopic Displays  Multi-Layer Displays - Layered 3D  Polarization Fields  Dual-Layer Displays - High-Rank 3D (HR3D)
    24. 24. Barco E-2320 PA Grayscale IPS LCD1600x1200 @ 60 Hz
    25. 25. Four Stacked Liquid Crystal Panels Two Crossed Polarizers
    26. 26. Overview of LCDs I vertical polarizer color filter array liquid crystal cells horizontal polarizer I0 backlight Malus’ Law Intensity Modulation with Liquid Crystal Cells I = I0 sin2 (q )
    27. 27. Extending Layered 3D to Multi-Layer LCDs Virtual Planes Design Optimization LCD 3 • Eliminate redundant polarizers  Sequentially-crossed design q LCD 2 x LCD 1 backlight 2D Light Field q x
    28. 28. Extending Layered 3D to Multi-Layer LCDs Virtual Planes Design Optimization LCD 3 • Eliminate redundant polarizers  Use sequentially-crossed q LCD 2 x • Exploit field-sequential color  0.33 = 2.7% brightness LCD 1 backlight 2D Light Field q x
    29. 29. Polarization Field Displays Virtual Planes Design Optimization LCD 3 • Eliminate redundant polarizers  Use sequentially-crossed q LCD 2 x • Exploit field-sequential color  0.33 = 2.7% brightness LCD 1 • Further optimize polarizers  Minimum is a crossed pair backlight 2D Light Field q x
    30. 30. Polarization Field Displays Virtual Planes LCD 3 f3 Image Formation q f2 K LCD 2 x Q(x, q ) = åfk (x, q ) k=1 f1 L(x, q ) = sin 2 (Q(x,q )) LCD 1 backlight Tomographic Synthesis 2D Light Field Q(x,q ) = ±sin-1 ( ) L(x, q ) mod p Q = Pf q argmin Q - Pf 2 2 fmin £f £fmax x
    31. 31. Tomographic Image Synthesis Projection Matrix Target Light Field LCD Pixel Values = * b=Ax
    32. 32. SART [Simultaneous Algebraic Reconstruction Technique] b=Ax pre-compute some weights initial guess ATv Ax update clamp
    33. 33. Efficient GPU Implementation Ax ATvForward Projection (Multiview Rendering) Back Projection (Projective Texture Mapping)
    34. 34. Polarization Field Displaysviewer moves right viewer moves down Stacked Polarization Input 4D Light Field Rotating Layers 90° 0° Optimized Rotation Angles for Each Layer
    35. 35. Polarization Field Displaysviewer moves right viewer moves down Input 4D Light Field Reconstruction Results
    36. 36. Outline Automultiscopic Displays  Multi-Layer Displays - Layered 3D - Polarization Fields  Dual-Layer Displays  High-Rank 3D (HR3D)
    37. 37. Input 4D Light Field
    38. 38. Parallax Barrier: Front Layer
    39. 39. Parallax Barrier: Rear Layer
    40. 40. Analysis of Parallax Barriers k L[i,k] i kg[k] f[i] i L[i,k] ` light box L[i, k] = f [i]× g[k] L[i, k ]  f [i]  g[k ]
    41. 41. Analysis of Parallax Barriers L[i,k] k g[k] i f[i] ` light box TKen Perlin et al. An Autosteroscopic Display. 2000. L[i, k] = å ft [i] Ä gt [k]Yunhee Kim et al. Electrically Movable Pinhole Arrays. 2007. t=1
    42. 42. Content-Adaptive Parallax Barriers L[i,k] G kg[k] i ~` f[i] F L light box ~ L  FG
    43. 43. Content-Adaptive Parallax Barriers G ~` L = F arg min L - FG W , for F, G ³ 0 2 F,G
    44. 44. Content-Adaptive Parallax Barrier: Front Layer
    45. 45. Content-Adaptive Parallax Barrier: Rear Layer
    46. 46. Simulation Results
    47. 47. Prototype High-Rank 3D (HR3D) Display http://cameraculture.media.mit.edu/byo3dMatthew Hirsch and Douglas Lanman. Build Your Own 3D Display. SIGGRAPH 2010, SIGGRAPH Asia 2010, SIGGRAPH 2011.
    48. 48. Experimental Results Time-Multiplexed Parallax Barrier High-Rank 3D (HR3D)
    49. 49. Outline Automultiscopic Displays  Multi-Layer Displays - Layered 3D - Polarization Fields  Dual-Layer Displays - High-Rank 3D (HR3D)
    50. 50. High-Rank 3D (HR3D) Layered 3D Polarization Fields www.hr3d.info www.layered3d.info tinyurl.com/polarization-fields BiDi Screen Tensor Displays www.bidiscreen.com tinyurl.com/tensordisplays

    ×