Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

CORNAR: Looking Around Corners using Trillion FPS Imaging

68,872 views

Published on

We have built a camera that can look around corners and beyond the line of sight. The camera uses light that travels from the object to the camera indirectly, by reflecting off walls or other obstacles, to reconstruct a 3D shape.

Published in: Technology, Business
  • Be the first to comment

CORNAR: Looking Around Corners using Trillion FPS Imaging

  1. 1. Raskar, Camera Culture, MIT Media Lab Computational Light Transport: CORNAR: Looking Around Corners Camera Culture using Trillion FPS Imaging Ramesh Raskar Ramesh Raskar MIT Media Lab http://raskar.info/cornar
  2. 2. What isaround the corner ?
  3. 3. Can you lookaround the corner ?
  4. 4. Can you lookaround the corner ?
  5. 5. Multi-path Analysis2nd Bounce 1st Bounce 3rd Bounce
  6. 6. Co-designing Optical and Digital Processing Computational Optics Light TransportPhoton Hacking Displays Sensors Computational Illumination Photography Signal Processing Computer Vision Machine Learning Bit Hacking
  7. 7. Multi-path Analysis2nd Bounce 1st Bounce 3rd Bounce
  8. 8. CORNAR: Femto-Photography FemtoFlash Trillion FPS camera With M Bawendi, MIT Chemistry Serious Sync Computational Optics•2012: 3D around a corner (NatureComm, Velten, et. al.)•2011: Material Sensing (Siggraph Asia, Naik, Zhao, Velten, Raskar, Bala)•2011: DARPA Young Faculty Award•2011: Motion Sensing (CVPR, Pandharkar, Velten, Bardagjy, Bawendi, Raskar)•2009: Hidden barcode (Kirmani, Hutchinson, Davis, Raskar, ICCV’2009)•2008: Indirect depth (Hirsch, Raskar)•2008: Transient Light Transport (Raskar, Davis, March 2008)
  9. 9. Inverting Light Transport Multiple Scattering Direct/Global[Seitz , Kutulakos, Matsushita 2005] [Nayar, Raskar et al 2006] [Atcheson et al 2008] [Kutulakos, Steger 2005] Dual Photography LIDAR [Sen et al 2005]
  10. 10. Collision avoidance, robot navigation, …
  11. 11. …, bronchoscopies, …
  12. 12. z S x L sOccluder Streak- camera C Laser B beam Echoes of Light
  13. 13. z S x L sOccluder Streak- camera C Laser B beam Echoes of Light
  14. 14. z S x L R s Occluder Streak- camera 3rd bounceStreak Photo C Laser B beam Echoes of Light
  15. 15. Multi-Dimensional Light Transport5-D Transport
  16. 16. Why Pico-second Resolution? ToF Diff = 0.15 mm s2 1cm s1Occluder 3rd bounce Streak- camera C p1 p2 1st bounce Curse of Pythagoras
  17. 17. z S x L R s Occluder Streak- camera 3rd bounceStreak Photo C Laser B beam Echoes of Light
  18. 18. Trillion FPS ToF Streak Tube = Inverse of CRT Very accurate sync1D camera: Single scan line stretched vertically in time ~2 ps resolution, 480 lines ~= 1 ns But for small samples in biochemistry
  19. 19. Time-ImageR Time Profile for a single pixel
  20. 20. Time Image of a single pointTime, ~2ns each row Space, 640 pixels Third Bounce (First bounce not shown)
  21. 21. 3D shape result from synthetic data Forward Reconstruction Invertibility Analysis Wavefront Non-linear Scene Priors Resolution andPropagation Inversion dimensions
  22. 22. Steady State 4D [Kajiya, 1986] [Seitz.., 2005]Impulse Response, 5D [Raskar and Davis, 2007]
  23. 23. Time Resolved Multi-path Imaging Scene withhidden elements Ultra fast illumination and camera 5D Raw Capture Time profiles Signal Proc.Photo, geometry, reflectance Novel light transport beyond models and inference line of sight algorithms → t 3D Time images
  24. 24. Third Bounce (First bounce not shown)
  25. 25. Third Bounce (First bounce not shown)
  26. 26. Third Bounce (First bounce not shown)
  27. 27. Third Bounce (First bounce not shown)
  28. 28. Third Bounce (First bounce not shown)
  29. 29. Photos from Streak CameraCapture Setup Hidden Scene
  30. 30. Photos from Streak CameraCapture Setup Hidden Scene Reconstruction Overlay
  31. 31. Hidden 3D Shape Space-time Photos
  32. 32. Motion beyond line of sightPandharkar, Velten, Bardagjy, Lawson, Bawendi, Raskar, CVPR 2011
  33. 33. BRDF (reflectance) fromSingle Viewpoint andTime ImagesNaik, Zhao, Velten, Raskar, Bala, (SIGGRAPH Asia 2011)
  34. 34. …, cardioscopies, …Participating Media
  35. 35. Space-timePhotos
  36. 36. TrillionFrames Per SecondImaginghttp://raskar.info/trillionfps
  37. 37. Each frame = ~2ps = 0.6 mm of Light Travel
  38. 38. Camera Time
  39. 39. raskar.info/trillionfpsCamera Time
  40. 40. World Time
  41. 41. raskar.info/trillionfps Lorentz transformation: distances, velocities, ordering
  42. 42. FemtoFlash Trillion FPS camera Serious Sync Computational OpticsPixel  Ray  Wave  Photons (ampli+phase) Steady State Transient Impulse/Step •2012: 3D around corner (Nature Comm, Velten et al) •2011: Material Sensing (Siggraph Asia, Naik, Zhao, Velten, Raskar, Bala) •2011: DARPA Young Faculty Award •2011: Motion Sensing (CVPR, Pandharkar, Velten, Bardagjy, Bawendi, Raskar) •2009: Hidden barcode (Kirmani, Hutchinson, Davis, Raskar, ICCV’2009) •2008: Indirect depth (Hirsch, Raskar) •2008: Transient Light Transport (Raskar, Davis, March 2008)
  43. 43. • Collaborations Welcome• Dataset Available• Propose Configurationsraskar(at)media.mit.eduhttp://www.media.mit.edu/~raskar/cornar/
  44. 44. http://raskar.info Femto-PhotographyLooking Around the Corner BRDF Detection Trillion FPS Movies Space-time Transforms

×