Making Invisible Visible, Ramesh Raskar Keynote at Embedded Vision 2019
Associate Professor, MIT Media Lab Ramesh Raskar is founder of the Camera Culture research group at the Massachusetts Institute of Technology (MIT) Media Lab and associate professor of Media Arts and Sciences at MIT. Raskar is the co-inventor of radical imaging solutions including femto-photography, an ultra-fast imaging camera that can see around corners, low-cost eye-care solutions for the developing world and a camera that allows users to read pages of a book without opening the cover. He is a pioneer in the fields of imaging, computer vision and machine learning. Raskar’s focus is on building interfaces between social systems and cyber-physical systems. These interfaces span research in physical (e.g., sensors, health-tech), digital (e.g., tools to enable keeping data private in distributed machine learning applications) and global (e.g., geomaps, autonomous mobility) domains. Recent inventions by Raskar’s team include transient imaging to look around a corner, a next-generation CAT-scan machine, imperceptible markers for motion capture, long-distance barcodes, touch + hover 3D interaction displays and new theoretical models to augment light fields to represent wave phenomena. Raskar has dedicated his career to linking the best of the academic and entrepreneurial worlds with young engineers, igniting a passion for impact inventing. Raskar seeks to catalyze change on a massive scale by launching platforms that empower inventors to create solutions to improve lives globally. Raskar has received the Lemelson Award, ACM SIGGRAPH Achievement Award, DARPA Young Faculty Award, Alfred P. Sloan Research Fellowship, TR100 Award from MIT Technology Review and Global Indus Technovator Award. He has worked on special research projects at Google [X] and Facebook and co-founded and advised several companies. He holds more than 80 US patents. Making the Invisible Visible: Within Our Bodies, the World Around Us, and Beyond
Recommended
More Related Content
Similar to Making Invisible Visible, Ramesh Raskar Keynote at Embedded Vision 2019
Similar to Making Invisible Visible, Ramesh Raskar Keynote at Embedded Vision 2019 (20)
More from Camera Culture Group, MIT Media Lab
More from Camera Culture Group, MIT Media Lab (20)
Recently uploaded
Recently uploaded (20)
Making Invisible Visible, Ramesh Raskar Keynote at Embedded Vision 2019
- 1. Making Invisible Visible Inside, Around and Beyond Ramesh Raskar Associate Professor MIT
- 3. Mohan et al, ‘Bokode’, Siggraph; Swedish et al, ‘EyeSelfie’, 2015
- 4. Invisible Predictors Mohan et al, ‘Bokode’, Siggraph; Swedish et al, ‘EyeSelfie’, 2015
- 5. Can you see around corners ?
- 7. Invisible DataData Utility Data Silo, Privacy
- 8. Raskar, Camera Culture, MIT Media Lab MIT Camera Culture Grp Summer course: tiny.cc/mitcourse
- 11. Light in Slow Motion .. 10 Billion x Slow
- 13. Velten et al, Siggraph 2013
- 14. Raw Data
- 15. Can you see around corners ?
- 16. 2nd Bounce Multi-path Analysis 1st Bounce 3rd Bounce 16
- 17. Femto-Camera Wall Door Hidden Mannequin Velten et al, Nature Communications 2012 Seeing Around Corners
- 19. Raskar, Camera Culture, MIT Media Lab
- 20. Motion beyond line of sight Pandharkar, Velten, Bardagjy, Lawson, Bawendi, Raskar, CVPR 2011 20
- 21. Collision avoidance … 21 DARPA REVEAL Program $28M
- 23. Optical Jumbled Brush Endoscope Heshmat, Nature SciRep16 Cellular resolution at 5mm NSF Moonshot FLIM Location behind Tissue Satat, Nature Comm 15 Satat, Nature SciRep 17 CT-scan in a Rickshaw Kadambi 17
- 24. 1mm Ballistic Limit Surface 100 um 1 mm 10 mm 100 mm 10 um Spatial Resolution 1 um 10 um 100 um OCT2/3 photon Confocal Computational Photoscatterography Depth ‘All Photon’ Imaging for Tissues NSF Expedition 2017-2022
- 25. Cellular Resolution In-vivo Imaging Scattering Depth Fluorescence Lifetime
- 26. Beat Diffraction Gated imaging to overcome ambient light ‘Negative light' via destructive interference inside any volume Focus at or ‘heat’ any voxel .. Conquer time .. • Seeing around corners • Fog/Closed book • Endoscopes/ Optical Brush • Fluorescence Lifetime
- 27. 27 Ordinary camera to see around corners?
- 28. Tancik, Satat, Raskar, Flash Photography for Data-Driven Hidden Scene Recovery28 Training data = Renderings 100k’s renderings, no real photos Transfer to real experiments
- 33. Sanchez, Heshmat, Reza, Romberg, Raskar 2015
- 35. Satat, Maeda, Tancik, Raskar, ICCP 2018 Seeing thru Fog
- 36. Fog + Object Model Normal distribution background photon Gamma distribution Object photon 𝑓𝑇 𝑡 = 𝑃 𝐵 𝑓𝑇 𝑡 𝐵 + 𝑃 𝑆 𝑓𝑇 𝑡 𝑆 depth and reflectance Photon Timing
- 37. Pamplona, Mohan, Oliveira, Raskar ‘NETRA’
- 38. 38 Spot Diagram on LCD Inverse of Shack-Hartmann wavefront sensor User interactively creates the Spot Diagram 1. Displace 25 spots with smart UI CellPhone LCD EyePiece 2. Displace spots till single dot perceived
- 39. EyeSelfie: Retinal Self-Imaging, Swedish et al, SIGGRAPH 2015 Raskar TEDMED 2013 Roesch, Swedish, Raskar, Clinical Ophthalmology 2017
- 41. Invisible DataData Utility Data Silo, Privacy
- 42. Silo, Quality, Talent Incentive, ?Social Good Privacy, Regulation, Trade Secrets Invisible Data/ Data Sharing Friction
- 43. Silo, Quality, Talent Incentive, ?Social Good Privacy, Regulation, Trade Secrets Invisible Data/ Data Sharing Friction a. Auto ML b. Split Learning c. Data Market
- 44. Pool ‘small’ data + Train ML Private ML No Exchange of Raw Patient Data Gupta, Raskar ‘Distributed training of deep neural network over several agents’, 2017 Server
- 45. a. Automating ML Published in International Conference on Learning Representations (2017) Baker, Gupta, Naik, Raskar, ICLR 2017 Teacher: RL Student: Supervised ML
- 46. Today’s Distributed Computing Server Client1 Client2 Client3 ..
- 47. Data Utility Train Model EncryptObfuscate Add Noise e.g. Differential Privacy Homomorphic Encryption Data Protect Infer Statistics Anonymize Training Deep Networks without exchanging raw patient data
- 48. Split Learning (MIT) ~Federated Learning Share Wisdom. Not Raw Data Data Utility Train Model EncryptSmashObfuscate Add Noise e.g. Differential Privacy Homomorphic Encryption Data Protect Infer Statistics Anonymize Training Deep Networks without exchanging raw patient data
- 49. Today’s Distributed Computing Server Client1 Client2 Client3 ..
- 50. Federated Learning Server Client1 Client2 Client3 ..
- 51. b. Split Learning Server Client1 Client2 Client3 .. Smasher Smashed Data Back Prop
- 52. 52 VGG over CIFAR 10 ResNet over CIFAR 100 Federated Split Compute Bandwidth Vepakomma, Swedish, Gupta, Dubey, Raskar 2018Raskar, McMahan et al CVPR 2019
- 53. Absolute Value Relative Value Conditional Value, additional users or features Intrinsic Goal-independent Value independent of the final goal Goal - specific Value based on ML algorithm Privacy-preserving Value without revealing raw data (with or without goal) Extrinsic Supply-Demand Speculated value using game theoretic multi-party interests Vepakomma, Swedish, Raskar 2019 c. Data Markets for ML
- 54. Silo, Quality, Talent Incentive, Privacy, Regulation, Trade Secrets Invisible Data/ Data Sharing Friction a. Auto ML b. Split Learning c. Data Market
- 55. Inside Tissue Imaging Data Markets ObjectsData Auto-ML CAT-scanEye Selfie FogBook Split Learning See around corners Making Invisible Visible | Summer course tiny.cc/mitcourse
- 56. Making Invisible Visible Inside, Around and Beyond Ramesh Raskar Associate Professor MIT