Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Multi-camera Time-of-Flight Systems | SIGGRAPH 2016

6,339 views

Published on

Computational imaging with multi-camera time-of-flight imaging systems.

Published in: Engineering
  • Be the first to comment

Multi-camera Time-of-Flight Systems | SIGGRAPH 2016

  1. 1. Computational Imaging with Multi-Camera Time-of-Flight Systems Felix Heide*1,2Shikhar Shrestha*1 Wolfgang Heidrich2,3 Gordon Wetzstein1 2University of British Columbia 3KAUST1Stanford University www.computationalimaging.org * joint first authors
  2. 2. Time-of-Flight Imaging Kinect for Xbox One Google Tango Phone Direct Sampling Transient Imaging [Velten et al. 2013] Google Self-Driving Car Facial capture in VR [Li et al. 2015] Correlation Transient Imaging [Heide et al. 2013]
  3. 3. Multi-Camera ToF System • Multiple Camera/Sources • Phased Array • Control over modulation • Extensible • Reproducible
  4. 4. light source camera Time-of-Flight ImagingTime-of-Flight Imaging
  5. 5. Image formation
  6. 6. Electron Field Photon Charge storage bin 1 Charge storage bin 2 Image formation
  7. 7. Image formation 1,2,3,4
  8. 8. Image formation 1,2,3,4 Static frame Captured with TI OPT8241 Slow-Motion Video Captured with TI OPT8241
  9. 9. Time-of-Flight ImagingTime-of-Flight Imaging – Dynamic Scene Doppler shift: Orthogonal Frequencies:
  10. 10. Doppler Shift
  11. 11. Image formation – Multicam
  12. 12. Image formation – Frequency and Phase Synchronization
  13. 13. Image formation – Frequency and Phase Synchronization
  14. 14. Multi-Camera ToF System • Exposure Synchronization • Individual Sensor/Source Modulation Signals • High Frequency Accuracy • Compatible with Recent ToF Cameras • Extensibility
  15. 15. Multi-Camera ToF System – High-Level System Architecture
  16. 16. Multi-Camera ToF System – Key System Components Direct Digital Synthesis (DDS) Ch 0 Ch 1 Ch 2 Ch 3 - Programmable Frequency/Phase - Resolution : 0.12 Hz/0.02° - Range : 0-200 MHz/0-360° - Control over SPI • Custom Board for End-to-End Signal Conditioning • Combines a High Speed Comparator with Exposure Gating IC • Interfaces Directly with TI ToF Camera Signal Conditioning Circuitry ToF Daughter BoardAD9959 Real-Time Controller STMF4 Discovery Board • Controller responsible for triggering frames, controlling waveform generation and synchronization • Runs low-level firmware based on an interrupt driven state machine architecture
  17. 17. Multi-Camera ToF System – System Workflow Signal Conditioning Circuitry (Now on ToF Daughter Board) DDS Block MCU Host Real-Time Controller Ref. Signal (Now uses SAW Oscillator @ 5—MHz) PC ToF Cameras
  18. 18. Multi-Camera ToF System – System Workflow Signal Conditioning Circuitry (Now on ToF Daughter Board) DDS Block Ref. Signal (Now uses SAW Oscillator @ 500 MHz) Configure Modulation Parameters ToF Cameras
  19. 19. Multi-Camera ToF System – System Workflow Signal Conditioning Circuitry (Now on ToF Daughter Board) DDS Block Ref. Signal (Now uses SAW Oscillator @ 5—MHz) Configure Camera Registers
  20. 20. Multi-Camera ToF System – System Workflow Signal Conditioning Circuitry (Now on ToF Daughter Board) DDS Block Ref. Signal (Now uses SAW Oscillator @ 5—MHz) Initiate a Frame
  21. 21. Multi-Camera ToF System – System Workflow DDS Block Ref. Signal (Now uses SAW Oscillator @ 5—MHz) Start of Micro-Exposure/Quad
  22. 22. Multi-Camera ToF System – System Workflow Enable Modulation
  23. 23. Multi-Camera ToF System – System Workflow Modulation Signals to Sensor/Illum
  24. 24. Multi-Camera ToF System – System Workflow Raw Capture Data
  25. 25. Multi-Camera ToF System – System Workflow
  26. 26. Multi-Camera ToF System – Demo Unit Design DC-DC Voltage Regulators Host MCUDDS Board ToF Daughter Board ToF Camera
  27. 27. Demo
  28. 28. Applications Multi-Camera Interference Parallel Phase Capture Doppler Velocity Capture Motion Non-Line-Of-Sight Motion in Scattering Media
  29. 29. Applications Multi-Camera Interference Parallel Phase Capture Doppler Velocity Capture Motion Non-Line-Of-Sight Motion in Scattering Media
  30. 30. Phased Camera Depth Imaging
  31. 31. Applications Multi-Camera Interference Parallel Phase Capture Doppler Velocity Capture Motion Non-Line-Of-Sight Motion in Scattering Media
  32. 32. Simultaneous Doppler Velocity Imaging
  33. 33. Simultaneous Doppler Velocity Imaging
  34. 34. Applications Multi-Camera Interference Parallel Phase Capture Doppler Velocity Capture Motion Non-Line-Of-Sight Motion in Scattering Media
  35. 35. Non-Line-of-Sight Motion Detection
  36. 36. Non-Line-of-Sight Motion Detection
  37. 37. Non-Line-of-Sight Motion Detection
  38. 38. Applications Multi-Camera Interference Parallel Phase Capture Doppler Velocity Capture Motion Non-Line-Of-Sight Motion in Scattering Media
  39. 39. Motion in Scattering Media
  40. 40. Motion in Scattering Media
  41. 41. Multi-Camera ToF System Open-Source Hardware Platform: www.computationalimaging.org

×