Image Destabilization ICCP 2009
•
2 likes•990 views
We propose a novel camera setup in which both the lens and the sensor are perturbed during the exposure. We analyze the defocus effects produced by such a setup, and use it to demonstrate new methods for simulating a lens with a larger effective aperture size (i.e., shallower depth of field) and methods for achieving approximately depth-independent defocus blur size. We achieve exaggerated, programmable, and pleasing bokeh with relatively small aperture sizes such as those found on cell phone cameras. Destabilizing the standard alignment of the sensor and lens allows us to introduce programmable defocus effects and achieve greater flexibility in the image capture process.
Recommended
More Related Content
More from Camera Culture Group, MIT Media Lab
More from Camera Culture Group, MIT Media Lab (20)
Recently uploaded
Recently uploaded (20)
Image Destabilization ICCP 2009
- 1. MIT Media Lab Camera Culture Image Destabilization: Programmable Defocus using Lens and Sensor Motion Ankit Mohan, Douglas Lanman, Shinsaku Hiura, Ramesh Raskar MIT Media Lab
- 2. MIT Media Lab Camera Culture Defocus Blur Lots of glass; Heavy; Bulky; Expensive
- 3. MIT Media Lab Camera Culture Image Destabilization Camera Lens Sensor Static Scene
- 4. MIT Media Lab Camera Culture Image Destabilization Camera Static Scene Lens Motion Sensor Motion
- 5. MIT Media Lab Camera Culture f8 Related Work f/4 f/2 [Bae and Durand 2007] extrapolated aperture f/1 [Hasinoff and Kutulakos 2007] [Vaish et al. 2004] [Hiura et al. 2009]
- 6. MIT Media Lab Camera Culture Laminography Motion direction X-Ray Source Plane of focus X-Ray Sensor Motion direction Related technique: Time Delay and Integration (TDI)
- 7. MIT Media Lab Camera Culture Lens based Focusing Lens Sensor A B’ A’ B
- 8. MIT Media Lab Camera Culture Lens based Focusing Lens Sensor A B’ A’ B
- 9. MIT Media Lab Camera Culture Smaller aperture Smaller defocus blur Lens Sensor A B’ A’ B
- 10. MIT Media Lab Camera Culture Pinhole: All In-Focus Pinhole Sensor A B’ A’ B
- 11. MIT Media Lab Camera Culture Shifting Pinhole Pinhole Sensor vp A B’ B A’
- 12. MIT Media Lab Camera Culture Shifting Pinhole Pinhole Sensor vp A B’ B A’
- 13. MIT Media Lab Camera Culture Shifting Pinhole Pinhole Sensor vp B’ A A’ B
- 14. MIT Media Lab Camera Culture Shifting Pinhole Pinhole Sensor vp B’ A A’ B
- 15. MIT Media Lab Camera Culture Shifting Pinhole Pinhole Sensor vp B’ A tp A’ B da db ds
- 16. MIT Media Lab Camera Culture Shifting Pinhole and Sensor Pinhole Sensor vp vs A B’ B A’ da db ds Focus Here
- 17. MIT Media Lab Camera Culture Shifting Pinhole and Sensor Pinhole Sensor vp vs B’ A A’ B da db ds Focus Here
- 18. MIT Media Lab Camera Culture Shifting Pinhole and Sensor Pinhole Sensor B’ vp vs A A’ B da db ds Focus Here
- 19. MIT Media Lab Camera Culture Shifting Pinhole and Sensor Pinhole Sensor B’ vp vs A A’ B da db ds Focus Here
- 20. MIT Media Lab Camera Culture A Lens in Time! Lens Equation: Virtual Focal Length: Virtual F-Number: Analogous to shift and sum based Light field re-focusing.
- 21. MIT Media Lab Camera Culture Our Prototype
- 22. MIT Media Lab Camera Culture Adjusting the Focus Plane all-in-focus pinhole image
- 23. MIT Media Lab Camera Culture Adjusting the Focus Plane focused in the front using destabilization (10 second exposure)
- 24. MIT Media Lab Camera Culture Adjusting the Focus Plane focused in the middle using destabilization (5 second exposure)
- 25. MIT Media Lab Camera Culture Adjusting the Focus Plane focused in the back using destabilization (10 second exposure)
- 26. MIT Media Lab Camera Culture Adjusting the Virtual Aperture focused in the middle using destabilization (5mm pinhole translation)
- 27. MIT Media Lab Camera Culture Adjusting the Virtual Aperture focused in the middle using destabilization (30mm pinhole translation)
- 28. MIT Media Lab Camera Culture Shifting Lens and Sensor Defocus Defocus Exaggeration • Physical vs. synthetic focus focus Similar physical and synthetic real focus virtual focus aperture sensor
- 29. MIT Media Lab Camera Culture Defocus Exaggeration static lens with an f/2.8 aperture
- 30. MIT Media Lab Camera Culture Defocus Exaggeration destabilization simulates a reduced f-number
- 31. MIT Media Lab Camera Culture Defocus Invariance • Differing physical and synthetic focus real focus virtual focus aperture sensor
- 32. MIT Media Lab Camera Culture Defocus Invariance • Related work – [Nagahara et al. 2008] – [Cathey and Dowski 1995] [Nagahara et al. 2008] – [Levin et al. 2008] • PSF not depth invariant * = – only size is depth invariant real PSF virtual PSF overall PSF • Gaussian special case – depth invariant PSF * = – inversion is ill-conditioned real PSF virtual PSF overall PSF
- 33. MIT Media Lab Camera Culture Defocus Invariance depth-invariant blur size (horizontal slit + destabilization)
- 34. MIT Media Lab Camera Culture Defocus Invariance Richardson-Lucy deconvolution result
- 35. MIT Media Lab Camera Culture Tilted Sensor focus plane aperture plane sensor plane dC C D′ D C′ dD d′D d′C Scheimpflug intersection
- 36. MIT Media Lab Camera Culture Tilted Sensor focus plane aperture plane sensor plane dC C vp vs? D dD
- 37. MIT Media Lab Camera Culture Tilted Sensor focus plane aperture plane sensor plane dC C vp vs D dD α α d′C d′D D′ C′
- 38. MIT Media Lab Camera Culture Tuning the PSF real focus aperture sensor pinhole image (static f/22 aperture)
- 39. MIT Media Lab Camera Culture Tuning the PSF real focus aperture sensor large aperture image (static f/2.8 aperture)
- 40. MIT Media Lab Camera Culture Tuning the PSF virtual focus aperture sensor destabilized image using a pinhole (translated f/22 aperture)
- 41. MIT Media Lab Camera Culture Tuning the PSF real focus virtual focus aperture sensor destabilized image using a large aperture (translated f/2.8 aperture)
- 42. MIT Media Lab Camera Culture Tuning the PSF real focus virtual focus aperture sensor simulated aspheric lens using a vertical slit aperture and destabilization
- 43. MIT Media Lab Camera Culture Extension to 2D Displacements linear circular elliptical “figure 8” hypocycloidal trispiral
- 44. MIT Media Lab Camera Culture Large apertures with tiny lenses? Benefits Limitations • No time or light inefficiency • Coordinated mechanical wrt cheap cameras movement required • Exploits unused area around • Diffraction (due to small aperture) the lens cannot be eliminated • Compact design [Zhang and Levoy, tomorrow] [Our group: augmented LF for wave analysis] • With near-pinhole apertures (mobile phones) many • Scene motion during exposure possibilities
- 45. MIT Media Lab Camera Culture Acknowledgements Grace Woo Quinn Smithwick Gabriel Taubin Jaewon Kim MIT CSAIL MIT Media Lab Brown University MIT Media Lab MIT Media Lab: Camera Culture
- 46. MIT Media Lab Camera Culture SLRs with tiny lenses? • Analysis of space of relative lens/sensor displacement • Destabilization as virtual focusing mechanism • Shallower depth of field than physical aperture • Depth-independent defocus blur size
Editor's Notes
- The technique was pioneered by radar engineer Edward Dowski and his thesis adviser Thomas Cathey at the University of Colorado in the United States in the 1990s. "Flexible Depth of Field Photography," H. Nagahara, S. Kuthirummal, C. Zhou, and S.K. Nayar, European Conference on Computer Vision (ECCV), Oct, 2008. Motion-Invariant Photography Anat Levin Peter Sand Taeg Sang Cho Fredo Durand William T. Freeman Computer Science and Artificial Intelligence Lab (CSAIL) Massachusetts Institute of Technology
- http://vismi.kaist.ac.kr/2004/research/APCBSolderJointInspectionusingDigitalTomosynthesis.htm http://www.e-radiography.net/radtech/t/tomo.htm