Download as PDF, PPTX
Uploaded byEdge AI and Vision Alliance
“Using a Collaborative Network of Distributed Cameras for Object Tracking,” a Presentation from Invision AI
The document outlines a collaborative network of distributed cameras designed for object tracking, focusing on scalability, privacy, and installation constraints. It discusses the mapping and calibration processes, tracking algorithms, and the system's ability to manage varying data rates while preserving efficiency. Remaining challenges include networking constraints, energy-efficient computation, and the automation of calibration processes.
More Related Content
Similar to “Using a Collaborative Network of Distributed Cameras for Object Tracking,” a Presentation from Invision AI
“Using a Collaborative Network of Distributed Cameras for Object Tracking,” a Presentation from Invision AI
- 1. Using a Collaborative Networkof Distributed Cameras for Object Tracking Samuel Orn Senior Machine Learning and Computer Vision Engineer Invision AI
- 2. Younge & College:8 Cameras Outdoors 2 © 2023 Invision AI
- 3. About Invision AI 3 ©2023 Invision AI Intersection Monitoring Vehicle Occupancy Detection Hazard Detection High-performance Road-side only Autonomous Rail Obstacle Detection
- 4. We want ourCollaborative Network of Distributed Cameras for Object Tracking to have the following properties: • Scalability with respect to the number of tracked objects • Scalability with respect to tracking area • Reasonable installation constraints (power and network requirements) • Privacy preserving Desired Properties of the System 4 © 2023 Invision AI
- 5. 5 © 2023 InvisionAI Part 1: Mapping and Calibration
- 6. Problems: 1. We needa map. 2. We need to calibrate with respect to the map. There are two different approaches of how to obtain maps and camera calibrations (camera positions and internal parameters); using structure from motion (SfM) and image matching, or finding already available maps and manually calibrate the cameras. Terrain Model and Calibration (Cont'd) 6 © 2023 Invision AI
- 7. Top-View and ElevationModel From Video 7 © 2023 Invision AI Acquisition Structure from motion point cloud (COLMAP, OpenSfm, OpenMVS) Top view + digital elevation model (DEM)
- 8. Calibration From Top-View+ DEM 8 © 2023 Invision AI Swiss coordinate system. Top-view + DEM publicly available. Calibration data links image coordinates with map coordinates.
- 9. 9 © 2023 InvisionAI Part 2: Tracking Algorithm
- 10. • Each processingnode is a Unix process. • Arrows represent inter process communication; can cross machine boundaries. • Only metadata leaves the deployment site. Image data only travels between camera and camera processor. System Overview 10 © 2023 Invision AI
- 11. • More processingpower is obtained by adding more processing nodes. • Can be on a new machine. • Multiple nodes of the same type gives us redundancy; if a processor node breaks (hardware or software), its load can be redistributed, and the system continues to work. System Overview: How to Scale 11 © 2023 Invision AI
- 12. • Trajectory estimatedas a spline. • 1 knot every second. • Pose (orientation and size) consistency across a track imposed through an on-line non-linear optimization process. • Able to deal with varying rates of incoming data. • Predicts motion in the future. Trajectory Representation 12 © 2023 Invision AI Cam2 frame Spline knot Prediction Motion over time Cam1 frame
- 13. Camera processor: • Produce2D observations (Neural Net object detector) and associate with trajectories. • Send association to trajectory processor. Trajectory processor: • Project observations (from all views) to common map coordinate system. • Fusion of all observations + motion model. • Send updated trajectory back to camera processors. From Observation to Trajectory 13 © 2023 Invision AI Trajectory Observation Match Project and incorporate in spline
- 14. 14 © 2023 InvisionAI Part 3: Demos
- 15. Demo, 8 CamerasIndoors 15 © 2023 Invision AI
- 16. Demo, 8 Cameras(Dashboard) 16 © 2023 Invision AI
- 17. 17 © 2023 InvisionAI Remaining Challenges and Conclusion
- 18. • Installation constraintsmeans it is not always possible to use a wired network: • Could we do networking with radio + mesh network? • The system needs a lot of compute; how to get a low energy, affordable NN inference system? • Anything else than NVidia Jetson products? • Calibration of the system is time consuming. Are there better solutions out there for automatic calibration? Questions and Challenges 18 © 2023 Invision AI
- 19.
- 20. References and resources Contactus! samuel.orn@invision.ai 20 © 2023 Invision AI Software and libraries COLMAP https://colmap.github.io/ OpenSfM https://opensfm.org/ PDAL https://pdal.io/