Raskar Mar09 Nesosa

1,163 views

Published on

Published in: Art & Photos, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,163
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
24
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Raskar Mar09 Nesosa

  1. 1. Raskar, Camera Culture, MIT Media Lab Computational Ph t C t ti l Photography: h Epsilon to Coded Imaging p g g Camera Culture Ramesh Raskar Camera Culture C C lt Associate Professor, MIT Media Lab http://raskar.info
  2. 2. Tools for Visual Computing Shado Refracti Reflecti w ve ve Fernald, Science [Sept 2006]
  3. 3. How can we create an entirely new class of imaging platforms that have an understanding of the world that far exceeds human ability and produce meaningful abstractions that are well within h ithi human comprehensibility ? h ibilit Ramesh Raskar http://raskar.info
  4. 4. Raskar 2006 Computational Illumination Augmented Reality Mitsubishi Electric Research Laboratories Spatial Planar Non planar Non-planar Curved Objects Pocket Proj Pocket-Proj 1998 1997 2002 2002 Single Projector Use r:T j ? Projector 1998 1998 2002 1999 2003 Multiple Projectors Computational Camera and Photography
  5. 5. Motion Blurred Photo
  6. 6. Short Sh t Traditional T diti l MURA Exposure Shutter Captured Single Photo Ph t Deblurred Result Banding Artifacts and Dark some spatial frequencies and noisy d i are lost
  7. 7. Blurring == Convolution Fourier Transform Sharp Sh Blurred Bl d Photo PSF == Sinc Function Photo ω Traditional Camera: Shutter is OPEN: Box Filter
  8. 8. Fourier Transform Sharp Sh Blurred Bl d Photo PSF == Broadband Function Photo Preserves High Spatial Frequencies Flutter Shutter: Shutter is OPEN and CLOSED
  9. 9. Flutter Shutter Camera Raskar, Agrawal, Tumblin [Siggraph2006] LCD opacity switched in coded sequence
  10. 10. Traditio Coded nal Exposu re Deblurred Deblurred Image I Image I Image of Static Object
  11. 11. Coded Exposure Coded Aperture Temporal 1-D broadband code: Spatial 2-D broadband mask: Motion Deblurring Focus Deblurring
  12. 12. Coded Aperture Camera The aperture of a 100 mm lens is modified Insert a coded mask with chosen binary pattern Rest of the camera is unmodified
  13. 13. LED In Focus Photo
  14. 14. Out of Focus Photo: Open Aperture
  15. 15. Out of Focus Photo: Coded Aperture
  16. 16. Captured Blurred Photo
  17. 17. Refocused on Person
  18. 18. Raskar, Camera Culture, MIT Media Lab Computational Photography 1. Epsilon Photography – Low-level Vision: Pixels – Multiphotos by bracketing (HDR, panorama) – ‘Ultimate camera’ 2. Coded Photography – Mid-Level Cues: • Regions, Edges, Motion, Direct/global g , g , , g – Single/few snapshot • Reversible encoding of data – Additional sensors/optics/illum 3. Essence Photography – Not mimic human eye – Beyond single view/illum – ‘New artform’
  19. 19. Raskar, Camera Culture, MIT Media Lab • Ramesh Raskar and Jack T bli J k Tumblin • Book Publishers: A K Peters
  20. 20. Less is More Blocking Light == More Information Coding in Time Coding in Space
  21. 21. Larval Trematode Worm Coded Aperture Camera
  22. 22. Shielding Light … g g Larval Trematode Worm Turbellarian Worm
  23. 23. Mask Sensor Mask ? Sensor Mask Full Resolution Digital 4D Light Field from Refocusing: 2D Photo: Coded Aperture Camera Heterodyne Light Field d h ld Camera
  24. 24. Light Field Inside a Camera
  25. 25. Light Field Inside a Camera Lenslet- Lenslet-based Light Field camera [Adelson and Wang, 1992, Ng et al. 2005 ]
  26. 26. Stanford Plenoptic Camera [Ng et al 2005] Contax medium format camera Kodak 16-megapixel sensor Adaptive Optics microlens array 125μ square-sided microlenses 4000 × 4000 pixels ÷ 292 × 292 lenses = 14 × 14 pixels per lens
  27. 27. Digital Refocusing g g [Ng et al 2005] Can we achieve this with a Mask alone?
  28. 28. Mask based Light Field Camera Sensor Mask [Veeraraghavan, Raskar, Agrawal, Tumblin, Mohan, Siggraph 2007 ]
  29. 29. How to Capture 4D Light Field with g 2D Sensor ? What h ld be th Wh t should b the pattern of the mask ?
  30. 30. Cosine Mask Used Mask Tile 1/f0
  31. 31. Captured 2D Photo Encoding due to Mask
  32. 32. Sensor Slice captures entire Light Field fθ fθ0 fx fx0 Modulation M d l i Function Modulated Light Field
  33. 33. Computing 4D Light Field 2D Sensor Photo, 1800*1800 2D Fourier Transform, 1800*1800 2D FFT 9*9=81 spectral copies Rearrange 2D tiles into 4D 4D IFFT 200*200*9*9 planes 4D Light Field 200*200*9*9
  34. 34. Captured Full resolution 2D image 2D Photo of Focused Scene Parts divide Image of White Lambertian Plane
  35. 35. Wavefront Sensing in Any Wavelength ! Sensor Mask [Veeraraghavan, Raskar, Agrawal, Tumblin, Mohan, Siggraph 2007 ]
  36. 36. Lens Flare Reduction/Enhancement using 4D Ray Sampling Glare Captured Glare Enhanced Reduced
  37. 37. Glare = low frequency noise in 2D •But is high frequency noise in 4D •Remove via simple outlier rejection Sensor i j u x
  38. 38. Rays = Waves for Propagation and Interface Fresnel propagation Chirp (Lens) Fourier transform Fractional Fourier transform x1 x2 x3 x4 x0 b ¡ a x0 u1 u2 u3 b u4 -ax b 0 x0 x1 x0 x2 x3 -bx a 0 x4 x0 - a -x a 0 I x4
  39. 39. Imaging via volume hologram (Depth-specific Imaging) KVH ( x =0, u =θ /λ; x , u ) 4 4 s 3 3 -20 u4 1 u3 -15 0.8 -10 0.6 -5 L u3 [mm-1] 0 0.4 x3 x4 5 0.2 10 0 15 20 -0.2 -0.4 -0.2 0 0.2 0.4 x3 [mm] ZZ ½ µ ¶¾ 0 0 µs dx 0 dx 0 e¡ i 2¼( u 4 x 4 ¡ u 3 x 3 ) K V H (x 4 ; u4 ; x 3 ; u3 ) =3 4 0 0 exp ¡ i 2¼ zf (u3 + u4 ) ¡ u3 + u4 ¡ ¸ ¸ ½ µ 0 0 ¶µ 0 ¶¾ ½ µ 0 0 ¶µ ¶¾ u3 + u4 u4 µs u3 + u4 u0 4 µs £ sinc L ¸ ¡ u3 + u4 + u4 + ¡ sinc L ¸ ¡ u3 + u4 ¡ u4 ¡ ¡ 2 2 ¸ 2 2 ¸ ½ ¾ ½ ¾ ¼ zf 2 L Derivation: h(x 2 ; x 1 ) = exp ¡ i 2 (x 1 + x 2 ¡ f µs) sinc (x 1 + x 2 ) (x 2 ¡ f µs) ¸ f ¸f2 K V H I (x 2 ; u2 ; x 1 ; u1 ) Parameters: ¸=05¹ 0.5 ¹m µs= 30° K V H (x 4 ; u4 ; x 3 ; u3 ) L = 1 mm zf = 50 mm
  40. 40. Camera Culture Group Raskar, Camera Culture, MIT Media Lab Ramesh Raskar http://raskar.info Computational Photography 1. Epsilon Photography Mask Sensor – Low-level Vision: Pixels – Multiphotos by bracketing (HDR, panorama) – ‘Ultimate camera’ 2. Coded Photography – Mid Level Mid-Level Cues: • Regions, Edges, Motion, Direct/global • Coded Exposure – Flutter Shutter Motion Deblurring • Coded Aperture – Defocus • Optical Heterodyning -20 1 – Lightfield or Wavefront sensing -15 -10 0.8 • Coded Glare 0.6 -5 u3 [mm-1] 0 0.4 • p y 6D Display 5 10 0.2 0 • Femto-second Imaging 15 2D 2D 2D 1 20 -0.2 2 -0.4 -0.2 0 0.2 0.4 x3 [mm] • Rays = Waves 1 1
  41. 41. How can we create an entirely new class of imaging platforms that have an understanding of the world that far exceeds human ability and produce meaningful abstractions that are well within h ithi human comprehensibility ? h ibilit Ramesh Raskar http://raskar.info

×