MIT Media Lab   Camera Culture




         Image Destabilization:

        Programmable Defocus using
           Lens and...
MIT Media Lab   Camera Culture


                         Defocus Blur

     Lots of glass; Heavy; Bulky; Expensive
MIT Media Lab    Camera Culture


                 Image Destabilization
                                            Camer...
MIT Media Lab    Camera Culture


                 Image Destabilization
                                                 ...
MIT Media Lab   Camera Culture
                                                                        f8

               ...
MIT Media Lab   Camera Culture


                        Laminography
                                 Motion direction
  ...
MIT Media Lab   Camera Culture


                Lens based Focusing
                                 Lens   Sensor



   ...
MIT Media Lab   Camera Culture


                Lens based Focusing
                                 Lens   Sensor



   ...
MIT Media Lab   Camera Culture


      Smaller aperture  Smaller defocus blur

                                 Lens   Se...
MIT Media Lab   Camera Culture


                Pinhole: All In-Focus
                                 Pinhole   Sensor

...
MIT Media Lab   Camera Culture


                      Shifting Pinhole
                                  Pinhole   Sensor...
MIT Media Lab   Camera Culture


                      Shifting Pinhole
                                  Pinhole   Sensor...
MIT Media Lab   Camera Culture


                      Shifting Pinhole
                                  Pinhole   Sensor...
MIT Media Lab   Camera Culture


                      Shifting Pinhole
                                  Pinhole   Sensor...
MIT Media Lab   Camera Culture


                      Shifting Pinhole
                                       Pinhole    ...
MIT Media Lab   Camera Culture


        Shifting Pinhole and Sensor
                                       Pinhole       ...
MIT Media Lab   Camera Culture


        Shifting Pinhole and Sensor
                                       Pinhole       ...
MIT Media Lab   Camera Culture


        Shifting Pinhole and Sensor
                                       Pinhole       ...
MIT Media Lab   Camera Culture


        Shifting Pinhole and Sensor
                                       Pinhole       ...
MIT Media Lab   Camera Culture


                      A Lens in Time!
                Lens Equation:


         Virtual F...
MIT Media Lab   Camera Culture


                        Our Prototype
MIT Media Lab   Camera Culture


          Adjusting the Focus Plane




                           all-in-focus pinhole i...
MIT Media Lab   Camera Culture


          Adjusting the Focus Plane




        focused in the front using destabilizatio...
MIT Media Lab   Camera Culture


          Adjusting the Focus Plane




        focused in the middle using destabilizati...
MIT Media Lab   Camera Culture


          Adjusting the Focus Plane




        focused in the back using destabilization...
MIT Media Lab   Camera Culture


       Adjusting the Virtual Aperture




     focused in the middle using destabilizatio...
MIT Media Lab   Camera Culture


       Adjusting the Virtual Aperture




     focused in the middle using destabilizatio...
MIT Media Lab      Camera Culture


     Shifting Lens and Sensor Defocus
            Defocus Exaggeration
   •  Physical ...
MIT Media Lab   Camera Culture


                Defocus Exaggeration




                    static lens with an f/2.8 ap...
MIT Media Lab   Camera Culture


                Defocus Exaggeration




            destabilization simulates a reduced ...
MIT Media Lab      Camera Culture


                     Defocus Invariance
   •  Differing physical and synthetic focus
 ...
MIT Media Lab    Camera Culture


                    Defocus Invariance
                                            •  Re...
MIT Media Lab   Camera Culture


                  Defocus Invariance




      depth-invariant blur size (horizontal slit...
MIT Media Lab   Camera Culture


                  Defocus Invariance




                 Richardson-Lucy deconvolution r...
MIT Media Lab     Camera Culture


                           Tilted Sensor
focus plane                         aperture p...
MIT Media Lab     Camera Culture


                           Tilted Sensor
focus plane                        aperture pl...
MIT Media Lab     Camera Culture


                           Tilted Sensor
focus plane                        aperture pl...
MIT Media Lab   Camera Culture


                      Tuning the PSF
real focus




aperture




sensor

                ...
MIT Media Lab   Camera Culture


                      Tuning the PSF
real focus




aperture




sensor

                ...
MIT Media Lab        Camera Culture


                            Tuning the PSF

virtual focus




aperture




sensor

 ...
MIT Media Lab       Camera Culture


                           Tuning the PSF
real focus
virtual focus




aperture




s...
MIT Media Lab    Camera Culture


                        Tuning the PSF
real focus




virtual focus



aperture




sens...
MIT Media Lab   Camera Culture


    Extension to 2D Displacements


                                   linear       circu...
MIT Media Lab   Camera Culture


      Large apertures with tiny lenses?
   Benefits                           Limitations...
MIT Media Lab    Camera Culture


                   Acknowledgements


     Grace Woo         Quinn Smithwick    Gabriel ...
MIT Media Lab   Camera Culture


                SLRs with tiny lenses?




   •    Analysis of space of relative lens/sen...
Upcoming SlideShare
Loading in …5
×

Image Destabilization ICCP 2009

1,993 views

Published on

We propose a novel camera setup in which both the lens and the sensor are perturbed during the exposure. We analyze the defocus effects produced by such a setup, and use it to demonstrate new methods for simulating a lens with a larger effective aperture size (i.e., shallower depth of field) and methods for achieving approximately depth-independent defocus blur size. We achieve exaggerated, programmable, and pleasing bokeh with relatively small aperture sizes such as those found on cell phone cameras. Destabilizing the standard alignment of the sensor and lens allows us to introduce programmable defocus effects and achieve greater flexibility in the image capture process.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,993
On SlideShare
0
From Embeds
0
Number of Embeds
1,168
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • The technique was pioneered by radar engineer Edward Dowski and his thesis adviser Thomas Cathey at the University of Colorado in the United States in the 1990s. "Flexible Depth of Field Photography," H. Nagahara, S. Kuthirummal, C. Zhou, and S.K. Nayar, European Conference on Computer Vision (ECCV), Oct, 2008. Motion-Invariant Photography Anat Levin   Peter Sand   Taeg Sang Cho   Fredo Durand   William T. Freeman Computer Science and Artificial Intelligence Lab (CSAIL) Massachusetts Institute of Technology
  • http://vismi.kaist.ac.kr/2004/research/APCBSolderJointInspectionusingDigitalTomosynthesis.htm http://www.e-radiography.net/radtech/t/tomo.htm
  • Image Destabilization ICCP 2009

    1. 1. MIT Media Lab Camera Culture Image Destabilization:
 Programmable Defocus using Lens and Sensor Motion Ankit Mohan, Douglas Lanman, Shinsaku Hiura, Ramesh Raskar MIT Media Lab
    2. 2. MIT Media Lab Camera Culture Defocus Blur Lots of glass; Heavy; Bulky; Expensive
    3. 3. MIT Media Lab Camera Culture Image Destabilization Camera Lens Sensor Static Scene
    4. 4. MIT Media Lab Camera Culture Image Destabilization Camera Static Scene Lens Motion Sensor Motion
    5. 5. MIT Media Lab Camera Culture f8 Related Work f/4 f/2 [Bae and Durand 2007] extrapolated aperture f/1 [Hasinoff and Kutulakos 2007] [Vaish et al. 2004] [Hiura et al. 2009]
    6. 6. MIT Media Lab Camera Culture Laminography Motion direction X-Ray Source Plane of focus X-Ray Sensor Motion direction Related technique: Time Delay and Integration (TDI)
    7. 7. MIT Media Lab Camera Culture Lens based Focusing Lens Sensor A B’ A’ B
    8. 8. MIT Media Lab Camera Culture Lens based Focusing Lens Sensor A B’ A’ B
    9. 9. MIT Media Lab Camera Culture Smaller aperture  Smaller defocus blur Lens Sensor A B’ A’ B
    10. 10. MIT Media Lab Camera Culture Pinhole: All In-Focus Pinhole Sensor A B’ A’ B
    11. 11. MIT Media Lab Camera Culture Shifting Pinhole Pinhole Sensor vp A B’ B A’
    12. 12. MIT Media Lab Camera Culture Shifting Pinhole Pinhole Sensor vp A B’ B A’
    13. 13. MIT Media Lab Camera Culture Shifting Pinhole Pinhole Sensor vp B’ A A’ B
    14. 14. MIT Media Lab Camera Culture Shifting Pinhole Pinhole Sensor vp B’ A A’ B
    15. 15. MIT Media Lab Camera Culture Shifting Pinhole Pinhole Sensor vp B’ A tp A’ B da db ds
    16. 16. MIT Media Lab Camera Culture Shifting Pinhole and Sensor Pinhole Sensor vp vs A B’ B A’ da db ds Focus Here
    17. 17. MIT Media Lab Camera Culture Shifting Pinhole and Sensor Pinhole Sensor vp vs B’ A A’ B da db ds Focus Here
    18. 18. MIT Media Lab Camera Culture Shifting Pinhole and Sensor Pinhole Sensor B’ vp vs A A’ B da db ds Focus Here
    19. 19. MIT Media Lab Camera Culture Shifting Pinhole and Sensor Pinhole Sensor B’ vp vs A A’ B da db ds Focus Here
    20. 20. MIT Media Lab Camera Culture A Lens in Time! Lens Equation: Virtual Focal Length: Virtual F-Number: Analogous to shift and sum based Light field re-focusing.
    21. 21. MIT Media Lab Camera Culture Our Prototype
    22. 22. MIT Media Lab Camera Culture Adjusting the Focus Plane all-in-focus pinhole image
    23. 23. MIT Media Lab Camera Culture Adjusting the Focus Plane focused in the front using destabilization (10 second exposure)
    24. 24. MIT Media Lab Camera Culture Adjusting the Focus Plane focused in the middle using destabilization (5 second exposure)
    25. 25. MIT Media Lab Camera Culture Adjusting the Focus Plane focused in the back using destabilization (10 second exposure)
    26. 26. MIT Media Lab Camera Culture Adjusting the Virtual Aperture focused in the middle using destabilization (5mm pinhole translation)
    27. 27. MIT Media Lab Camera Culture Adjusting the Virtual Aperture focused in the middle using destabilization (30mm pinhole translation)
    28. 28. MIT Media Lab Camera Culture Shifting Lens and Sensor Defocus Defocus Exaggeration •  Physical vs. synthetic focus focus Similar physical and synthetic real focus virtual focus aperture sensor
    29. 29. MIT Media Lab Camera Culture Defocus Exaggeration static lens with an f/2.8 aperture
    30. 30. MIT Media Lab Camera Culture Defocus Exaggeration destabilization simulates a reduced f-number
    31. 31. MIT Media Lab Camera Culture Defocus Invariance •  Differing physical and synthetic focus real focus virtual focus aperture sensor
    32. 32. MIT Media Lab Camera Culture Defocus Invariance •  Related work –  [Nagahara et al. 2008] –  [Cathey and Dowski 1995] [Nagahara et al. 2008] –  [Levin et al. 2008] •  PSF not depth invariant * = –  only size is depth invariant real PSF virtual PSF overall PSF •  Gaussian special case –  depth invariant PSF * = –  inversion is ill-conditioned real PSF virtual PSF overall PSF
    33. 33. MIT Media Lab Camera Culture Defocus Invariance depth-invariant blur size (horizontal slit + destabilization)
    34. 34. MIT Media Lab Camera Culture Defocus Invariance Richardson-Lucy deconvolution result
    35. 35. MIT Media Lab Camera Culture Tilted Sensor focus plane aperture plane sensor plane dC C D′ D C′ dD d′D d′C Scheimpflug intersection
    36. 36. MIT Media Lab Camera Culture Tilted Sensor focus plane aperture plane sensor plane dC C vp vs? D dD
    37. 37. MIT Media Lab Camera Culture Tilted Sensor focus plane aperture plane sensor plane dC C vp vs D dD α α d′C d′D D′ C′
    38. 38. MIT Media Lab Camera Culture Tuning the PSF real focus aperture sensor pinhole image (static f/22 aperture)
    39. 39. MIT Media Lab Camera Culture Tuning the PSF real focus aperture sensor large aperture image (static f/2.8 aperture)
    40. 40. MIT Media Lab Camera Culture Tuning the PSF virtual focus aperture sensor destabilized image using a pinhole (translated f/22 aperture)
    41. 41. MIT Media Lab Camera Culture Tuning the PSF real focus virtual focus aperture sensor destabilized image using a large aperture (translated f/2.8 aperture)
    42. 42. MIT Media Lab Camera Culture Tuning the PSF real focus virtual focus aperture sensor simulated aspheric lens using a vertical slit aperture and destabilization
    43. 43. MIT Media Lab Camera Culture Extension to 2D Displacements linear circular elliptical “figure 8” hypocycloidal trispiral
    44. 44. MIT Media Lab Camera Culture Large apertures with tiny lenses? Benefits Limitations •  No time or light inefficiency •  Coordinated mechanical wrt cheap cameras movement required •  Exploits unused area around •  Diffraction (due to small aperture) the lens cannot be eliminated •  Compact design [Zhang and Levoy, tomorrow] [Our group: augmented LF for wave analysis] •  With near-pinhole apertures (mobile phones) many •  Scene motion during exposure possibilities
    45. 45. MIT Media Lab Camera Culture Acknowledgements Grace Woo Quinn Smithwick Gabriel Taubin Jaewon Kim MIT CSAIL MIT Media Lab Brown University MIT Media Lab MIT Media Lab: Camera Culture
    46. 46. MIT Media Lab Camera Culture SLRs with tiny lenses? •  Analysis of space of relative lens/sensor displacement •  Destabilization as virtual focusing mechanism •  Shallower depth of field than physical aperture •  Depth-independent defocus blur size

    ×