Introduction to Light Fields
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
784
On Slideshare
784
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
41
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Put diagrams here
  • Sub aperture views

Transcript

  • 1. Introduction to Light Fields
  • 2. The 5D Plenoptic Function Q: What is the set of all things that one can ever see? A: The Plenoptic Function [Adelson and Bergen 1991] (from plenus, complete or full, and optic) P(q, f, l, t)
  • 3. The 5D Plenoptic Function Q: What is the set of all things that one can ever see? A: The Plenoptic Function [Adelson and Bergen 1991] (from plenus, complete or full, and optic) P(q, f, l, t)
  • 4. Position-Angle Parameterization 2D position 2D direction s q
  • 5. Two-Plane Parameterization 2D position 2D position s u
  • 6. Two-Plane Parameterization – camera array 2D position 2D position s u
  • 7. Two-Plane Parameterization – SLR camera 2D position 2D position s u
  • 8. Two-Plane Parameterization – cell phone s u
  • 9. Two-Plane Parameterization – cell phone (pelican) s u
  • 10. Application: Digital Image Refocusing [Ng 2005] Kodak 16-megapixel sensor 125μ square-sided microlenses
  • 11. Application: 3D Displays Parallax Panoramagram [Kanolt 1918] 3DTV with Integral Imaging [Okano et al. 1999] MERL 3DTV [Matusik and Pfister 2004]
  • 12. Multiple Sensors
  • 13. Static Camera Arrays Stanford Multi-Camera Array 125 cameras using custom hardware [Wilburn et al. 2002, Wilburn et al. 2005] Distributed Light Field Camera 64 cameras with distributed rendering [Yang et al. 2002]
  • 14. Refocus
  • 15. Temporal Multiplexing
  • 16. Controlled Camera or Object Motion Stanford Spherical Gantry [Levoy and Hanrahan 1996] Relighting with 4D Incident Light Fields [Masselus et al. 2003]
  • 17. Uncontrolled Camera or Object Motion Unstructured Lumigraph Rendering [Gortler et al. 1996; Buehler et al. 2001]
  • 18. Spatial Multiplexing
  • 19. Parallax Barriers (Pinhole Arrays) [Ives 1903] sensor barrier Spatially-multiplexed light field capture using masks (i.e., barriers): • Cause severe attenuation  long exposures or lower SNR • Impose fixed trade-off between spatial and angular resolution (unless implemented with programmable masks, e.g. LCDs)
  • 20. Integral Imaging (“Fly’s Eye” Lenslets) [Lippmann 1908] sensor lenslet f Spatially-multiplexed light field capture using lenslets: • Impose fixed trade-off between spatial and angular resolution
  • 21. Light Field Photograph (Sensor)
  • 22. Light Field Photograph (Decoded) [The (New) Stanford Light Field Archive] lookingup looking to the right Sample Image
  • 23. DEMO lenstoys: light field camera
  • 24. Modern, Digital Implementations Digital Light Field Photography • Hand-held plenoptic camera [Ng et al. 2005] • Heterodyne light field camera [Veeraraghavan et al. 2007]
  • 25.   Marc Levoy  Light Field = Array of (Virtual) Cameras Slide by Marc Levoy
  • 26.   Marc Levoy  Sub-aperture Virtual Camera = Sub-aperture View Light Field = Array of (Virtual) Cameras Slide by Marc Levoy
  • 27.   Marc Levoy  Sub-aperture Virtual Camera = Sub-aperture View Light Field = Array of (Virtual) Cameras Slide by Marc Levoy
  • 28.   Marc Levoy
  • 29. DEMO http://lightfield.stanford.edu/aperture.swf?lightfield=data/chess_lf/preview.zip&zoom=1
  • 30. The Lytro Light Field Camera
  • 31. Stanford camera array
  • 32. Microlens camera array Digital sensor Microlens array (MLA)
  • 33. Lytro: microlens array in camera Main lens Microlens array (MLA) Digital sensor Lens is not really thin, but can be treated as so. 330 × 330 hex array, 13.9 micron pitch Occluders are not required. 14 Mpixel, square cropped to 11 Mpixels Choose pixel in same location behind each microlens
  • 34. Microlens array (MLA) Main lens Digital sensor # of sub-apertures = # of pixels behind each microlens (10 × 10) # of pixels per sub-aperture image = # of microlenses (~ 120,000) All rays pass through a “sub-aperture” Sub-aperture captures on camera view Choose pixel in same location behind each microlens So why put the microlens array inside the camera?
  • 35. Lytro camera has unusual shape 8x f/2 lens Light field sensor Battery 43-343 mm equivalent
  • 36. Other Applications of Light Fields
  • 37. Lens Glare Reduction [Raskar, Agrawal, Wilson, Veeraraghavan SIGGRAPH 2008] Glare/Flare due to camera lenses reduces contrast
  • 38. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Reducing Glare Glare Reduced Image After removing outliersConventional Photo
  • 39. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Enhancing Glare Glare Enhanced ImageConventional Photo
  • 40. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan
  • 41. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan
  • 42. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Glare due to Lens Inter-Reflections a Sensor b
  • 43. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Effects of Glare on Image • Hard to model, Low Frequency in 2D • But reflection glare is outlier in 4D ray-space Angular Variation at pixel a Lens Inter-reflections a Sensor b
  • 44. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Key Idea • Lens Glare manifests as low frequency in 2D Image • But Glare is highly view dependent – manifests as outliers in 4D ray-space • Reducing Glare == Remove outliers among rays a Sensor b
  • 45. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Reducing Glare using a Light Field Camera
  • 46. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Single Shot Light Field Cameras Using Mask, this paper Using Lenslets, Ng et al. 2005 Mask Adelson and Wang, 1992, Ng et al. 2005 Kanolt 1933, Veeraraghavan et al. 2007
  • 47. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Captured Photo: LED off
  • 48. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Captured Photo: LED On
  • 49. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Each Disk: Angular Samples at that Spatial Location No Glare
  • 50. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan With Glare
  • 51. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan x y u v
  • 52. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Sequence of Sub-Aperture Views Average of all the Light Field views One of the Light Field views Low Res Traditional Camera Photo Glare Reduced Image
  • 53. MERL, MIT Media Lab Glare Aware Photography: 4D Ray Sampling for Reducing Glare Raskar, Agrawal, Wilson & Veeraraghavan Key Idea • Reducing Glare == Remove outlier among angular samples a Sensor b
  • 54. Light-sensitive Displays Depth Cameras Multi-touch Interaction Multi-touch Interaction with Thin Displays
  • 55. BiDi Screen: Thin, Depth-sensing LCDs • Seamless transition from multi-touch to gesture • Thin form factor (LCD)
  • 56. LCD Modifications
  • 57. Diffuser Camera LCD Lights Prototype