Raskar Mar09 Nesosa
•
1 like•550 views
1. Ramesh Raskar is an associate professor at the MIT Media Lab researching computational photography. 2. Raskar discusses three levels of computational photography - epsilon, coded, and essence photography. Coded photography uses single or few snapshots but introduces reversible encoding of light through techniques like coded exposure and coded apertures. 3. Examples of coded photography techniques presented include flutter shutter motion deblurring, coded aperture defocus, optical heterodyning for lightfield or wavefront sensing, and using a coded glare mask. The goal is to create new imaging capabilities beyond what is possible with traditional cameras.
More Related Content
What's hot
What's hot (19)
Viewers also liked
Viewers also liked (7)
Similar to Raskar Mar09 Nesosa
Similar to Raskar Mar09 Nesosa (20)
More from Camera Culture Group, MIT Media Lab
More from Camera Culture Group, MIT Media Lab (20)
Recently uploaded
Recently uploaded (19)
Raskar Mar09 Nesosa
- 1. Raskar, Camera Culture, MIT Media Lab Computational Ph t C t ti l Photography: h Epsilon to Coded Imaging p g g Camera Culture Ramesh Raskar Camera Culture C C lt Associate Professor, MIT Media Lab http://raskar.info
- 3. Tools for Visual Computing Shado Refracti Reflecti w ve ve Fernald, Science [Sept 2006]
- 4. How can we create an entirely new class of imaging platforms that have an understanding of the world that far exceeds human ability and produce meaningful abstractions that are well within h ithi human comprehensibility ? h ibilit Ramesh Raskar http://raskar.info
- 5. Raskar 2006 Computational Illumination Augmented Reality Mitsubishi Electric Research Laboratories Spatial Planar Non planar Non-planar Curved Objects Pocket Proj Pocket-Proj 1998 1997 2002 2002 Single Projector Use r:T j ? Projector 1998 1998 2002 1999 2003 Multiple Projectors Computational Camera and Photography
- 8. Short Sh t Traditional T diti l MURA Exposure Shutter Captured Single Photo Ph t Deblurred Result Banding Artifacts and Dark some spatial frequencies and noisy d i are lost
- 9. Blurring == Convolution Fourier Transform Sharp Sh Blurred Bl d Photo PSF == Sinc Function Photo ω Traditional Camera: Shutter is OPEN: Box Filter
- 10. Fourier Transform Sharp Sh Blurred Bl d Photo PSF == Broadband Function Photo Preserves High Spatial Frequencies Flutter Shutter: Shutter is OPEN and CLOSED
- 11. Flutter Shutter Camera Raskar, Agrawal, Tumblin [Siggraph2006] LCD opacity switched in coded sequence
- 12. Traditio Coded nal Exposu re Deblurred Deblurred Image I Image I Image of Static Object
- 14. Coded Exposure Coded Aperture Temporal 1-D broadband code: Spatial 2-D broadband mask: Motion Deblurring Focus Deblurring
- 15. Coded Aperture Camera The aperture of a 100 mm lens is modified Insert a coded mask with chosen binary pattern Rest of the camera is unmodified
- 17. Out of Focus Photo: Open Aperture
- 18. Out of Focus Photo: Coded Aperture
- 19. Captured Blurred Photo
- 20. Refocused on Person
- 21. Raskar, Camera Culture, MIT Media Lab Computational Photography 1. Epsilon Photography – Low-level Vision: Pixels – Multiphotos by bracketing (HDR, panorama) – ‘Ultimate camera’ 2. Coded Photography – Mid-Level Cues: • Regions, Edges, Motion, Direct/global g , g , , g – Single/few snapshot • Reversible encoding of data – Additional sensors/optics/illum 3. Essence Photography – Not mimic human eye – Beyond single view/illum – ‘New artform’
- 22. Raskar, Camera Culture, MIT Media Lab • Ramesh Raskar and Jack T bli J k Tumblin • Book Publishers: A K Peters
- 23. Less is More Blocking Light == More Information Coding in Time Coding in Space
- 24. Larval Trematode Worm Coded Aperture Camera
- 25. Shielding Light … g g Larval Trematode Worm Turbellarian Worm
- 26. Mask Sensor Mask ? Sensor Mask Full Resolution Digital 4D Light Field from Refocusing: 2D Photo: Coded Aperture Camera Heterodyne Light Field d h ld Camera
- 27. Light Field Inside a Camera
- 28. Light Field Inside a Camera Lenslet- Lenslet-based Light Field camera [Adelson and Wang, 1992, Ng et al. 2005 ]
- 29. Stanford Plenoptic Camera [Ng et al 2005] Contax medium format camera Kodak 16-megapixel sensor Adaptive Optics microlens array 125μ square-sided microlenses 4000 × 4000 pixels ÷ 292 × 292 lenses = 14 × 14 pixels per lens
- 30. Digital Refocusing g g [Ng et al 2005] Can we achieve this with a Mask alone?
- 31. Mask based Light Field Camera Sensor Mask [Veeraraghavan, Raskar, Agrawal, Tumblin, Mohan, Siggraph 2007 ]
- 32. How to Capture 4D Light Field with g 2D Sensor ? What h ld be th Wh t should b the pattern of the mask ?
- 33. Cosine Mask Used Mask Tile 1/f0
- 34. Captured 2D Photo Encoding due to Mask
- 35. Sensor Slice captures entire Light Field fθ fθ0 fx fx0 Modulation M d l i Function Modulated Light Field
- 36. Computing 4D Light Field 2D Sensor Photo, 1800*1800 2D Fourier Transform, 1800*1800 2D FFT 9*9=81 spectral copies Rearrange 2D tiles into 4D 4D IFFT 200*200*9*9 planes 4D Light Field 200*200*9*9
- 37. Captured Full resolution 2D image 2D Photo of Focused Scene Parts divide Image of White Lambertian Plane
- 38. Wavefront Sensing in Any Wavelength ! Sensor Mask [Veeraraghavan, Raskar, Agrawal, Tumblin, Mohan, Siggraph 2007 ]
- 39. Lens Flare Reduction/Enhancement using 4D Ray Sampling Glare Captured Glare Enhanced Reduced
- 40. Glare = low frequency noise in 2D •But is high frequency noise in 4D •Remove via simple outlier rejection Sensor i j u x
- 41. Rays = Waves for Propagation and Interface Fresnel propagation Chirp (Lens) Fourier transform Fractional Fourier transform x1 x2 x3 x4 x0 b ¡ a x0 u1 u2 u3 b u4 -ax b 0 x0 x1 x0 x2 x3 -bx a 0 x4 x0 - a -x a 0 I x4
- 42. Imaging via volume hologram (Depth-specific Imaging) KVH ( x =0, u =θ /λ; x , u ) 4 4 s 3 3 -20 u4 1 u3 -15 0.8 -10 0.6 -5 L u3 [mm-1] 0 0.4 x3 x4 5 0.2 10 0 15 20 -0.2 -0.4 -0.2 0 0.2 0.4 x3 [mm] ZZ ½ µ ¶¾ 0 0 µs dx 0 dx 0 e¡ i 2¼( u 4 x 4 ¡ u 3 x 3 ) K V H (x 4 ; u4 ; x 3 ; u3 ) =3 4 0 0 exp ¡ i 2¼ zf (u3 + u4 ) ¡ u3 + u4 ¡ ¸ ¸ ½ µ 0 0 ¶µ 0 ¶¾ ½ µ 0 0 ¶µ ¶¾ u3 + u4 u4 µs u3 + u4 u0 4 µs £ sinc L ¸ ¡ u3 + u4 + u4 + ¡ sinc L ¸ ¡ u3 + u4 ¡ u4 ¡ ¡ 2 2 ¸ 2 2 ¸ ½ ¾ ½ ¾ ¼ zf 2 L Derivation: h(x 2 ; x 1 ) = exp ¡ i 2 (x 1 + x 2 ¡ f µs) sinc (x 1 + x 2 ) (x 2 ¡ f µs) ¸ f ¸f2 K V H I (x 2 ; u2 ; x 1 ; u1 ) Parameters: ¸=05¹ 0.5 ¹m µs= 30° K V H (x 4 ; u4 ; x 3 ; u3 ) L = 1 mm zf = 50 mm
- 43. Camera Culture Group Raskar, Camera Culture, MIT Media Lab Ramesh Raskar http://raskar.info Computational Photography 1. Epsilon Photography Mask Sensor – Low-level Vision: Pixels – Multiphotos by bracketing (HDR, panorama) – ‘Ultimate camera’ 2. Coded Photography – Mid Level Mid-Level Cues: • Regions, Edges, Motion, Direct/global • Coded Exposure – Flutter Shutter Motion Deblurring • Coded Aperture – Defocus • Optical Heterodyning -20 1 – Lightfield or Wavefront sensing -15 -10 0.8 • Coded Glare 0.6 -5 u3 [mm-1] 0 0.4 • p y 6D Display 5 10 0.2 0 • Femto-second Imaging 15 2D 2D 2D 1 20 -0.2 2 -0.4 -0.2 0 0.2 0.4 x3 [mm] • Rays = Waves 1 1
- 44. How can we create an entirely new class of imaging platforms that have an understanding of the world that far exceeds human ability and produce meaningful abstractions that are well within h ithi human comprehensibility ? h ibilit Ramesh Raskar http://raskar.info