4D Capture and 6D Display<br />Ankit Mohan<br />MIT Media Lab<br />
Light Field [Levoy & Hanrahan 1996]<br />Radiance as a function of position and direction<br />5D Plenoptic function (x, y...
4D Illumination Field<br />Measure all the incoming light rays<br />
4D x 4D = 8D Reflectance Field<br />Ratio:Rij  = (outgoing rayi) / (incoming rayj)<br />
Geometric Optics basedHigh Dimensional Capture & Display<br />4D capture: 4D outgoing rays<br />6D display: 4D outgoing ra...
Light Field Inside a Camera<br />s<br />u<br />Lenslet-based Light Field camera / Integral Photography<br />s<br />u<br />...
Adaptive Optics microlens array<br />125μ square-sided microlenses<br />Stanford Plenoptic Camera [Ng et al 2005]<br />Con...
Digital  Refocusing[Ng et al 2005]<br />Can we do this without the microlens array?<br />
Coded Aperture Camera<br />Lens aperture is modified<br />Broadband coded mask<br />
LED<br />In Focus Photo<br />
Out of Focus Photo: Open Aperture<br />
Out of Focus Photo: Coded Aperture<br />
Captured Blurred Photo<br />
Refocused on Person<br />
Mask?<br />Mask?<br />Sensor<br />Sensor<br />Sensor<br />Sensor<br />Mask<br />Mask<br />Mask?<br />Sensor<br />4D Light ...
Sensor<br />Mask<br />Mask based Light Field Camera<br />
Optical Heterodyning<br />Receiver: Demodulation<br />High Freq Carrier 100 MHz<br />Incoming Signal<br />Baseband Audio S...
Captured 2D Photo<br />Encoding due to Mask<br />
Heterodyning – Frequency domain<br />2D FFT<br />Traditional Camera Photo<br />Magnitude of 2D FFT<br />2D FFT<br />Hetero...
Computing 4D Light Field<br />2D Sensor Photo, 1800*1800<br />2D Fourier Transform, 1800*1800<br />2D FFT<br />9*9=81 spec...
4D Capture Results<br />Captured Photo<br />Refocusing<br />Changing Views<br />
Illumination Dependent Display<br />
Martin Fuchs &lt;mfuchs@mpi-inf.mpg.de&gt;<br />
4D Display Optical Setup<br />
Towards a 6D Display<br />
6D Display Prototype<br />
6D building block: light sensitive projector<br />
6D Display Optical Setup<br />
6D modulator pattern structure<br />
6D modulator pattern structure<br />varying observer direction<br />
6D Display Results<br />
Credits and Acknowledgements<br /><ul><li>Heterodyne camera: Ashok Veeraraghavan, AmitAgrawal, Jack Tumblin, RameshRaskar
6D Display: Martin Fuchs, Hans-Peter Seidel, Hendrik P. A. Lensch, RameshRaskar
Slides courtesy: Marc Levoy, Jack Tumblin, Martin Fuchs, RameshRaskar</li></li></ul><li>We focus on creating tools to bett...
Upcoming SlideShare
Loading in …5
×

Mit Museum Talk

1,883 views

Published on

Talk at MIT Museum workshop "Photons, Neurons and Bits: Holography for the 21st Century"

Published in: Education, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,883
On SlideShare
0
From Embeds
0
Number of Embeds
15
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • live demonstration: show 4D hand-held / bottle with light bulb source. After it is demonstrated, hand it round, then continue on how to do this.
  • end of presentation
  • TO DO TO DO TO DOKann man da was highlighten beim erklären?now, we can add some blockers and modulate; the design you currently see. One deficit: the cones on the receiver surface do not stay below the lens, and do not have a precise overlap
  • This is what the 6D prototype looks like …
  • This is one of 49 “single pixel” blocks for the 6D display. It projects out different patterns according to the incident light illumination.To test it separately, we feed it with a special pattern
  • Luckily, we can add more lenses. This rotates the beam back, which makes it a translation operation.
  • Mit Museum Talk

    1. 1. 4D Capture and 6D Display<br />Ankit Mohan<br />MIT Media Lab<br />
    2. 2. Light Field [Levoy & Hanrahan 1996]<br />Radiance as a function of position and direction<br />5D Plenoptic function (x, y, z, , f)<br /><ul><li>4D in free space (u, v, s, t)</li></li></ul><li>4D Light Field<br />Measure all the outgoing light rays.<br />
    3. 3. 4D Illumination Field<br />Measure all the incoming light rays<br />
    4. 4. 4D x 4D = 8D Reflectance Field<br />Ratio:Rij = (outgoing rayi) / (incoming rayj)<br />
    5. 5. Geometric Optics basedHigh Dimensional Capture & Display<br />4D capture: 4D outgoing rays<br />6D display: 4D outgoing rays + 2D incoming rays<br />
    6. 6. Light Field Inside a Camera<br />s<br />u<br />Lenslet-based Light Field camera / Integral Photography<br />s<br />u<br />[Lippman 1908, Adelsonand Wang, 1992, Ng et al. 2005]<br />
    7. 7. Adaptive Optics microlens array<br />125μ square-sided microlenses<br />Stanford Plenoptic Camera [Ng et al 2005]<br />Contax medium format camera<br />Kodak 16-megapixel sensor<br />4000 × 4000 pixels ÷ 292 × 292 lenses = 14 × 14 pixels per lens<br />
    8. 8. Digital Refocusing[Ng et al 2005]<br />Can we do this without the microlens array?<br />
    9. 9. Coded Aperture Camera<br />Lens aperture is modified<br />Broadband coded mask<br />
    10. 10. LED<br />In Focus Photo<br />
    11. 11. Out of Focus Photo: Open Aperture<br />
    12. 12. Out of Focus Photo: Coded Aperture<br />
    13. 13. Captured Blurred Photo<br />
    14. 14. Refocused on Person<br />
    15. 15. Mask?<br />Mask?<br />Sensor<br />Sensor<br />Sensor<br />Sensor<br />Mask<br />Mask<br />Mask?<br />Sensor<br />4D Light Field from 2D Photo: <br />Heterodyne Light Field Camera<br />Full Resolution Digital Refocusing:<br />Coded Aperture Camera<br />
    16. 16. Sensor<br />Mask<br />Mask based Light Field Camera<br />
    17. 17. Optical Heterodyning<br />Receiver: Demodulation<br />High Freq Carrier 100 MHz<br />Incoming Signal<br />Baseband Audio Signal<br />ReferenceCarrier<br />99 MHz<br />Software Demodulation<br />Main Lens<br />Object<br />Mask<br />Sensor<br />RecoveredLight Field<br />Incident Modulated Signal<br />Photographic Signal(Light Field)<br />Carrier <br />ReferenceCarrier<br />
    18. 18. Captured 2D Photo<br />Encoding due to Mask<br />
    19. 19. Heterodyning – Frequency domain<br />2D FFT<br />Traditional Camera Photo<br />Magnitude of 2D FFT<br />2D FFT<br />Heterodyne Camera Photo<br />Magnitude of 2D FFT<br />
    20. 20. Computing 4D Light Field<br />2D Sensor Photo, 1800*1800<br />2D Fourier Transform, 1800*1800<br />2D FFT<br />9*9=81 spectral copies<br />Rearrange 2D tiles into 4D planes<br />4D IFFT<br />200*200*9*9<br />4D Light Field<br />200*200*9*9<br />
    21. 21. 4D Capture Results<br />Captured Photo<br />Refocusing<br />Changing Views<br />
    22. 22. Illumination Dependent Display<br />
    23. 23. Martin Fuchs &lt;mfuchs@mpi-inf.mpg.de&gt;<br />
    24. 24.
    25. 25. 4D Display Optical Setup<br />
    26. 26. Towards a 6D Display<br />
    27. 27. 6D Display Prototype<br />
    28. 28. 6D building block: light sensitive projector<br />
    29. 29. 6D Display Optical Setup<br />
    30. 30. 6D modulator pattern structure<br />
    31. 31. 6D modulator pattern structure<br />varying observer direction<br />
    32. 32. 6D Display Results<br />
    33. 33. Credits and Acknowledgements<br /><ul><li>Heterodyne camera: Ashok Veeraraghavan, AmitAgrawal, Jack Tumblin, RameshRaskar
    34. 34. 6D Display: Martin Fuchs, Hans-Peter Seidel, Hendrik P. A. Lensch, RameshRaskar
    35. 35. Slides courtesy: Marc Levoy, Jack Tumblin, Martin Fuchs, RameshRaskar</li></li></ul><li>We focus on creating tools to better capture and share visual information <br />The goal is to create an entirely new class of imaging platforms that have an understanding of the world that far exceeds human ability and produce meaningful abstractions that are well within human comprehensibility<br />Ramesh Raskar http://raskar.info<br />

    ×