TARDEC Presentation 2

391 views
329 views

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
391
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

TARDEC Presentation 2

  1. 1. Technology OverviewChris SlaughterPresidentProf. Sriram VishwanathChief Scientist
  2. 2. Our Team 2 Prof. Sriram Vishwanath • 9 years, Prof., UT Austin • Information Theory, Entrepreneurship • 2 Robotics Labs: MA coordination and 3D Perception Chris Slaughter • PhD. Candidate, Electrical Engineering • 1.5 years consulting T.O. of Austin Startup • Research Lead, UT Perception Laboratory Ongoing Collaborations/Partnerships • Lockheed Martin • HKS • NVIDIA • UC Berkeley
  3. 3. Our Team 3• Computer Vision: Multi-view geometry and stereo; tracking; 3d reconstruction• High Performance Computing: General purpose graphics programming (GPGPU), parallel/distributed computing, heterogeneous computation• Statistics and Learning: Large scale clustering problems, compressive motion analysis, graphical inference• Embedded Systems: Board design, multi-processor interaction, interfaces, power/weight/form factor
  4. 4. Mission Statement 4 To Teach Unmanned Vehicles to See as Humans Do Applications: • Absolute localization in GPS denied scenarios • Visual tracking odometry • Landmark detection and landmark-based navigation • Terrain mapping and change detection for IED disposal • Immersive visualization for situational awareness • Disaster response
  5. 5. System Architecture 4 Server Nodes Producer Nodes Consumer Nodes
  6. 6. System Architecture 5 Server Nodes Producer Nodes Consumer Nodes
  7. 7. Producer Nodes
  8. 8. Producer Nodes 7 Computing Element: Embedded GPGPU processor ARM multi-core Heterogeneous architecture Optoelectronic Device: Active Stereo (IR) Passive Stereo Laser / TOF
  9. 9. Producer Nodes 8 Key Features: • Visual Odometry ( < 3 mm ) • Mapping • Landmark Extraction • High Data Rates ( 9.2M pts / sec ) • Volumetric / Manifold Reconstruction • Source Compression for Uplink Compatibility: • Low power ( < 4.5 W ) • Low weight ( cell phone – battery ) • Low cost ( COTS sensors )
  10. 10. Producer Nodes 9 Signal Processing • Fundamental task in vision-based algorithms • Most algorithms too slow even for desktops • Bilateral filters • Pyramid computation • Depth-RGB conversion • SIFT descriptors • Object recognition
  11. 11. Image Filtering
  12. 12. Pyramid Computation
  13. 13. Frame Matching
  14. 14. Producer Nodes 1 3 Visual Odometry • RGB-based • 30 FPS • Accuracy in mm • Compressive sensing solution • Range-based • 9.2M pts / sec • 90T comps / sec • Towards drift free?
  15. 15. Vision-Only Tracking
  16. 16. RANSAC (1983)
  17. 17. RANSAC (1983) SOLO (2010)
  18. 18. Producer Nodes 1 7 Can a mobile device map 3D environments to produce dense 3D data points rather than sparse landmarks? Dense Reconstruction • Landmark-based mapping (1960’s – present) – SOLVED • DARPA grand challenge • Efficient global alignment (1999 – 2005) – SOLVED • Multiple view geometry + dense reconstruction (1980’s – present) • Live dense reconstruction (2006 – present) • KinectFusion • Range-based SLAM • Dense Tracking and Mapping
  19. 19. Volumetric Reconstruction
  20. 20. Point Cloud Reconstruction
  21. 21. Patchwork Reconstruction
  22. 22. Depth Data
  23. 23. Producer Nodes 2 2 Patchwork Reconstructions • More memory efficient than volumes • Faster integration and tracking • Efficient caching interplay • Global refinement • Runs on embedded device
  24. 24. Server Nodes
  25. 25. Server Nodes 2 4 Global Refinement • Maps must be globally consistent • Dense reconstruction doesn’t allow for this refinement • Patchwork generalizes to this functionality • Inherently multi-core problem • ARM architecture for GraDeS
  26. 26. No RefinementHOGMAN (2007)
  27. 27. Full Visualization 2 6 Custom Visualization • Key task: visualization • Interactivity (XBOX controller) • Full support for 3d / 2d streams • Event processing • Interoperability with GPGPU (CUDA) • Compression for consumer nodes • Fully scalable Rendering Pipeline • Custom rendering pipeline for Patchwork • Volumetric and Point Cloud • Raycasting with lighting sources
  28. 28. Consumer Nodes
  29. 29. Consumer Nodes 2 8 Our technology can produce maps at high speeds and unprecedented fidelities But.. What to do with this content? Visual Localization Situational Awareness• Localization a major problem in GPS- • Visualize mapping assets in real time denied scenarios from cell phones • “Urban canyons” • Coordinate with server and receive • Indoor environments compressed video stream • MAV / UGV coordination • Back-end models dynamics of• Existing solutions based mostly on adversaries state estimation • Extensible visualizer: new• Possible to query large maps for tags, models, data sources location?
  30. 30. Visual Absolute Localization
  31. 31. Feasible Path Infeasible Path
  32. 32. Positional Decoding
  33. 33. Conclusion 3 2 Current trends in computer vision and robotics: • High performance computing • Live dense reconstruction • Range-based tracking and mapping Our architecture: • Producer nodes: • COTS sensors • Commodity computational unit • Dense tracking and mapping • Server nodes • Combine producer data into large maps • Serve consumer nodes • Consumer nodes • Visual absolute localization and remote visualization
  34. 34. Thanks!

×