Introduction to Light Fields

3,278 views

Published on

Published in: Art & Photos, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,278
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
175
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • Put diagrams here
  • Sub aperture views
  • Introduction to Light Fields

    1. 1. Introduction to Light Fields
    2. 2. The 5D Plenoptic Function Q: What is the set of all things that one can ever see? A: The Plenoptic Function [Adelson and Bergen 1991] (from plenus, complete or full, and optic) P(q, f, l, t)
    3. 3. The 5D Plenoptic Function Q: What is the set of all things that one can ever see? A: The Plenoptic Function [Adelson and Bergen 1991] (from plenus, complete or full, and optic) P(q, f, l, t)
    4. 4. Position-Angle Parameterization 2D position 2D direction s q
    5. 5. Two-Plane Parameterization 2D position 2D position s u
    6. 6. Two-Plane Parameterization – camera array 2D position 2D position s u
    7. 7. Two-Plane Parameterization – SLR camera 2D position 2D position s u
    8. 8. Two-Plane Parameterization – cell phone s u
    9. 9. Two-Plane Parameterization – cell phone (pelican) s u
    10. 10. Application: Digital Image Refocusing [Ng 2005] Kodak 16-megapixel sensor 125μ square-sided microlenses
    11. 11. Application: 3D Displays Parallax Panoramagram [Kanolt 1918] 3DTV with Integral Imaging [Okano et al. 1999] MERL 3DTV [Matusik and Pfister 2004]
    12. 12. Multiple Sensors
    13. 13. Static Camera Arrays Stanford Multi-Camera Array 125 cameras using custom hardware [Wilburn et al. 2002, Wilburn et al. 2005] Distributed Light Field Camera 64 cameras with distributed rendering [Yang et al. 2002]
    14. 14. Refocus
    15. 15. Temporal Multiplexing
    16. 16. Controlled Camera or Object Motion Stanford Spherical Gantry [Levoy and Hanrahan 1996] Relighting with 4D Incident Light Fields [Masselus et al. 2003]
    17. 17. Uncontrolled Camera or Object Motion Unstructured Lumigraph Rendering [Gortler et al. 1996; Buehler et al. 2001]
    18. 18. Spatial Multiplexing
    19. 19. Parallax Barriers (Pinhole Arrays) [Ives 1903] sensor barrier Spatially-multiplexed light field capture using masks (i.e., barriers): • Cause severe attenuation  long exposures or lower SNR • Impose fixed trade-off between spatial and angular resolution (unless implemented with programmable masks, e.g. LCDs)
    20. 20. Integral Imaging (“Fly’s Eye” Lenslets) [Lippmann 1908] sensor lenslet f Spatially-multiplexed light field capture using lenslets: • Impose fixed trade-off between spatial and angular resolution
    21. 21. Light Field Photograph (Sensor)
    22. 22. Light Field Photograph (Decoded) [The (New) Stanford Light Field Archive] lookingup looking to the right Sample Image
    23. 23. DEMO lenstoys: light field camera
    24. 24. Modern, Digital Implementations Digital Light Field Photography • Hand-held plenoptic camera [Ng et al. 2005] • Heterodyne light field camera [Veeraraghavan et al. 2007]
    25. 25.   Marc Levoy  Light Field = Array of (Virtual) Cameras Slide by Marc Levoy
    26. 26.   Marc Levoy  Sub-aperture Virtual Camera = Sub-aperture View Light Field = Array of (Virtual) Cameras Slide by Marc Levoy
    27. 27.   Marc Levoy  Sub-aperture Virtual Camera = Sub-aperture View Light Field = Array of (Virtual) Cameras Slide by Marc Levoy
    28. 28.   Marc Levoy
    29. 29. DEMO http://lightfield.stanford.edu/aperture.swf?lightfield=data/chess_lf/preview.zip&zoom=1
    30. 30. The Lytro Light Field Camera
    31. 31. Stanford camera array
    32. 32. Microlens camera array Digital sensor Microlens array (MLA)
    33. 33. Lytro: microlens array in camera Main lens Microlens array (MLA) Digital sensor Lens is not really thin, but can be treated as so. 330 × 330 hex array, 13.9 micron pitch Occluders are not required. 14 Mpixel, square cropped to 11 Mpixels Choose pixel in same location behind each microlens
    34. 34. Microlens array (MLA) Main lens Digital sensor # of sub-apertures = # of pixels behind each microlens (10 × 10) # of pixels per sub-aperture image = # of microlenses (~ 120,000) All rays pass through a “sub-aperture” Sub-aperture captures on camera view Choose pixel in same location behind each microlens So why put the microlens array inside the camera?
    35. 35. Lytro camera has unusual shape 8x f/2 lens Light field sensor Battery 43-343 mm equivalent
    36. 36. Other Applications of Light Fields
    37. 37. Lens Glare Reduction [Raskar, Agrawal, Wilson, Veeraraghavan SIGGRAPH 2008] Glare/Flare due to camera lenses reduces contrast
    38. 38. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Reducing Glare Glare Reduced Image After removing outliersConventional Photo
    39. 39. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Enhancing Glare Glare Enhanced ImageConventional Photo
    40. 40. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan
    41. 41. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan
    42. 42. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Glare due to Lens Inter-Reflections a Sensor b
    43. 43. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Effects of Glare on Image • Hard to model, Low Frequency in 2D • But reflection glare is outlier in 4D ray-space Angular Variation at pixel a Lens Inter-reflections a Sensor b
    44. 44. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Key Idea • Lens Glare manifests as low frequency in 2D Image • But Glare is highly view dependent – manifests as outliers in 4D ray-space • Reducing Glare == Remove outliers among rays a Sensor b
    45. 45. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Reducing Glare using a Light Field Camera
    46. 46. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Single Shot Light Field Cameras Using Mask, this paper Using Lenslets, Ng et al. 2005 Mask Adelson and Wang, 1992, Ng et al. 2005 Kanolt 1933, Veeraraghavan et al. 2007
    47. 47. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Captured Photo: LED off
    48. 48. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Captured Photo: LED On
    49. 49. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Each Disk: Angular Samples at that Spatial Location No Glare
    50. 50. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan With Glare
    51. 51. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan x y u v
    52. 52. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Sequence of Sub-Aperture Views Average of all the Light Field views One of the Light Field views Low Res Traditional Camera Photo Glare Reduced Image
    53. 53. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Key Idea • Reducing Glare == Remove outlier among angular samples a Sensor b
    54. 54. Light-sensitive Displays Depth Cameras Multi-touch Interaction Multi-touch Interaction with Thin Displays
    55. 55. BiDi Screen: Thin, Depth-sensing LCDs • Seamless transition from multi-touch to gesture • Thin form factor (LCD)
    56. 56. LCD Modifications
    57. 57. Diffuser Camera LCD Lights Prototype

    ×