Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Making Invisible Visible, Ramesh Raskar Keynote at Embedded Vision 2019

589 views

Published on

Associate Professor, MIT Media Lab
Ramesh Raskar is founder of the Camera Culture research group at the Massachusetts Institute of Technology (MIT) Media Lab and associate professor of Media Arts and Sciences at MIT. Raskar is the co-inventor of radical imaging solutions including femto-photography, an ultra-fast imaging camera that can see around corners, low-cost eye-care solutions for the developing world and a camera that allows users to read pages of a book without opening the cover. He is a pioneer in the fields of imaging, computer vision and machine learning.

Raskar’s focus is on building interfaces between social systems and cyber-physical systems. These interfaces span research in physical (e.g., sensors, health-tech), digital (e.g., tools to enable keeping data private in distributed machine learning applications) and global (e.g., geomaps, autonomous mobility) domains. Recent inventions by Raskar’s team include transient imaging to look around a corner, a next-generation CAT-scan machine, imperceptible markers for motion capture, long-distance barcodes, touch + hover 3D interaction displays and new theoretical models to augment light fields to represent wave phenomena.

Raskar has dedicated his career to linking the best of the academic and entrepreneurial worlds with young engineers, igniting a passion for impact inventing. Raskar seeks to catalyze change on a massive scale by launching platforms that empower inventors to create solutions to improve lives globally.

Raskar has received the Lemelson Award, ACM SIGGRAPH Achievement Award, DARPA Young Faculty Award, Alfred P. Sloan Research Fellowship, TR100 Award from MIT Technology Review and Global Indus Technovator Award. He has worked on special research projects at Google [X] and Facebook and co-founded and advised several companies. He holds more than 80 US patents.

Making the Invisible Visible: Within Our Bodies, the World Around Us, and Beyond

Published in: Technology

Making Invisible Visible, Ramesh Raskar Keynote at Embedded Vision 2019

  1. 1. Making Invisible Visible Inside, Around and Beyond Ramesh Raskar Associate Professor MIT
  2. 2. Mohan et al, ‘Bokode’, Siggraph; Swedish et al, ‘EyeSelfie’, 2015
  3. 3. Invisible Predictors Mohan et al, ‘Bokode’, Siggraph; Swedish et al, ‘EyeSelfie’, 2015
  4. 4. Can you see around corners ?
  5. 5. Invisible Objects
  6. 6. Invisible DataData Utility Data Silo, Privacy
  7. 7. Raskar, Camera Culture, MIT Media Lab MIT Camera Culture Grp Summer course: tiny.cc/mitcourse
  8. 8. Milli Micro Nano Pico Femto Atto Conquer .. Time
  9. 9. Milli Micro Nano Pico Atto Femto-graphy Conquer .. Time Noise Signal
  10. 10. Light in Slow Motion .. 10 Billion x Slow
  11. 11. Velten et al, Siggraph 2013
  12. 12. Raw Data
  13. 13. Can you see around corners ?
  14. 14. 2nd Bounce Multi-path Analysis 1st Bounce 3rd Bounce 16
  15. 15. Femto-Camera Wall Door Hidden Mannequin Velten et al, Nature Communications 2012 Seeing Around Corners
  16. 16. Raskar, Camera Culture, MIT Media Lab
  17. 17. Motion beyond line of sight Pandharkar, Velten, Bardagjy, Lawson, Bawendi, Raskar, CVPR 2011 20
  18. 18. Collision avoidance … 21 DARPA REVEAL Program $28M
  19. 19. Femto-Photography Endoscope 22
  20. 20. Optical Jumbled Brush Endoscope Heshmat, Nature SciRep16 Cellular resolution at 5mm NSF Moonshot FLIM Location behind Tissue Satat, Nature Comm 15 Satat, Nature SciRep 17 CT-scan in a Rickshaw Kadambi 17
  21. 21. 1mm Ballistic Limit Surface 100 um 1 mm 10 mm 100 mm 10 um Spatial Resolution 1 um 10 um 100 um OCT2/3 photon Confocal Computational Photoscatterography Depth ‘All Photon’ Imaging for Tissues NSF Expedition 2017-2022
  22. 22. Cellular Resolution In-vivo Imaging Scattering Depth Fluorescence Lifetime
  23. 23. Beat Diffraction Gated imaging to overcome ambient light ‘Negative light' via destructive interference inside any volume Focus at or ‘heat’ any voxel .. Conquer time .. • Seeing around corners • Fog/Closed book • Endoscopes/ Optical Brush • Fluorescence Lifetime
  24. 24. 27 Ordinary camera to see around corners?
  25. 25. Tancik, Satat, Raskar, Flash Photography for Data-Driven Hidden Scene Recovery28 Training data = Renderings 100k’s renderings, no real photos Transfer to real experiments
  26. 26. Invisible Objects
  27. 27. Sanchez, Heshmat, Reza, Romberg, Raskar 2015
  28. 28. Satat, Maeda, Tancik, Raskar, ICCP 2018 Seeing thru Fog
  29. 29. Fog + Object Model Normal distribution background photon Gamma distribution Object photon 𝑓𝑇 𝑡 = 𝑃 𝐵 𝑓𝑇 𝑡 𝐵 + 𝑃 𝑆 𝑓𝑇 𝑡 𝑆 depth and reflectance Photon Timing
  30. 30. Pamplona, Mohan, Oliveira, Raskar ‘NETRA’
  31. 31. 38 Spot Diagram on LCD Inverse of Shack-Hartmann wavefront sensor User interactively creates the Spot Diagram 1. Displace 25 spots with smart UI CellPhone LCD EyePiece 2. Displace spots till single dot perceived
  32. 32. EyeSelfie: Retinal Self-Imaging, Swedish et al, SIGGRAPH 2015 Raskar TEDMED 2013 Roesch, Swedish, Raskar, Clinical Ophthalmology 2017
  33. 33. Invisible DataData Utility Data Silo, Privacy
  34. 34. Silo, Quality, Talent Incentive, ?Social Good Privacy, Regulation, Trade Secrets Invisible Data/ Data Sharing Friction
  35. 35. Silo, Quality, Talent Incentive, ?Social Good Privacy, Regulation, Trade Secrets Invisible Data/ Data Sharing Friction a. Auto ML b. Split Learning c. Data Market
  36. 36. Pool ‘small’ data + Train ML Private ML No Exchange of Raw Patient Data Gupta, Raskar ‘Distributed training of deep neural network over several agents’, 2017 Server
  37. 37. a. Automating ML Published in International Conference on Learning Representations (2017) Baker, Gupta, Naik, Raskar, ICLR 2017 Teacher: RL Student: Supervised ML
  38. 38. Today’s Distributed Computing Server Client1 Client2 Client3 ..
  39. 39. Data Utility Train Model EncryptObfuscate Add Noise e.g. Differential Privacy Homomorphic Encryption Data Protect Infer Statistics Anonymize Training Deep Networks without exchanging raw patient data
  40. 40. Split Learning (MIT) ~Federated Learning Share Wisdom. Not Raw Data Data Utility Train Model EncryptSmashObfuscate Add Noise e.g. Differential Privacy Homomorphic Encryption Data Protect Infer Statistics Anonymize Training Deep Networks without exchanging raw patient data
  41. 41. Today’s Distributed Computing Server Client1 Client2 Client3 ..
  42. 42. Federated Learning Server Client1 Client2 Client3 ..
  43. 43. b. Split Learning Server Client1 Client2 Client3 .. Smasher Smashed Data Back Prop
  44. 44. 52 VGG over CIFAR 10 ResNet over CIFAR 100 Federated Split Compute Bandwidth Vepakomma, Swedish, Gupta, Dubey, Raskar 2018Raskar, McMahan et al CVPR 2019
  45. 45. Absolute Value Relative Value Conditional Value, additional users or features Intrinsic Goal-independent Value independent of the final goal Goal - specific Value based on ML algorithm Privacy-preserving Value without revealing raw data (with or without goal) Extrinsic Supply-Demand Speculated value using game theoretic multi-party interests Vepakomma, Swedish, Raskar 2019 c. Data Markets for ML
  46. 46. Silo, Quality, Talent Incentive, Privacy, Regulation, Trade Secrets Invisible Data/ Data Sharing Friction a. Auto ML b. Split Learning c. Data Market
  47. 47. Inside Tissue Imaging Data Markets ObjectsData Auto-ML CAT-scanEye Selfie FogBook Split Learning See around corners Making Invisible Visible | Summer course tiny.cc/mitcourse
  48. 48. Making Invisible Visible Inside, Around and Beyond Ramesh Raskar Associate Professor MIT

×