FØCAL Boston AiR - Computer Vision Tracing and Hardware Simulation
•
1 like•152 views
FØCAL presentation at Boston AIR (AI, Autonomy & Robotics) meetup on 28 Mar 2019. Challenges of getting computer vision applications from prototype to production. Image processing and computer vision pipeline tracing and edge device hardware selection platform.
More Related Content
What's hot
What's hot (20)
Similar to FØCAL Boston AiR - Computer Vision Tracing and Hardware Simulation
Similar to FØCAL Boston AiR - Computer Vision Tracing and Hardware Simulation (20)
Recently uploaded
Recently uploaded (20)
FØCAL Boston AiR - Computer Vision Tracing and Hardware Simulation
- 1. @ Boston AiR March 28, 2019
- 2. Education Bachelors Computer science, math, neuroscience University of Miami, 2001 Masters Computational vision Drs. Ennio Mingolla & Gail Carpenter Boston University, 2004 Models of human primary visual cortex applied to ground water detection in SAR imagery (NIMA) Career DARPA URGENT “AI for LIDAR” Vision engineer - LM/ATL + Stanford, Berkeley, Numenta DARPA Mind’s Eye “Grand Challenge” of CV Tech lead - MIT/Co57 team Consulting 5 years helping companies at all scales figure out how to leverage computer vision
- 3. ▼ Application + Sensors Software Hardware Architecture = Accuracy QoS SWaP Cost Prototype ▼ Data Algorithms + Accuracy =
- 4. ▼ Application + Sensors Software Hardware Architecture = Accuracy QoS SWaP Cost Prototype ▼ Data Algorithms + Accuracy = ?
- 5. ▼ Application + Sensors Software Hardware Architecture = Accuracy QoS SWaP Cost Prototype ▼ Data Algorithms + Accuracy = Profiling HW selection
- 6. LTTng Linux Tracing Toolkit -- next generation Unified user- and kernel-space runtime instrumentation. Empirical methods for prioritizing system optimizations. Problem → prescription is still very challenging for the naive. Conan Package manager for development workflows. “pip for C++.” Distributes binaries; zero-effort compilation on demand. Clever hashing system records the provenance of the artifacts. Community
- 7. $ python3 -m venv _venv $ source _venv/bin/activate $ pip install conan $ conan install opencv/4.0.1@conan/stable Commands and features subject to change - check latest documentation
- 8. LTTng Linux Tracing Toolkit -- next generation Unified user- and kernel-space runtime instrumentation. Empirical methods for prioritizing system optimizations. Problem → prescription is still very challenging for the naive. Conan Package manager for development workflows. “pip for C++.” Distributes binaries; zero-effort compilation on demand. Clever hashing system records the provenance of the artifacts. Community
- 9. $ curl -o https://bootstrap.f0cal.com -L bootstrap-f0cal $ python3 bootstrap-f0cal git -v _venv37 $ source _venv37/bin/activate $ f0cal hw add my_laptop --detect $ f0cal sw install opencv/4.0.1 $ git clone https://github.com/learnopencv/learnopencv.git $ cd learnopencv/MultiObjectTracker $ f0cal sw build ./ $ f0cal pr add -- ./bin/multitracker $ f0cal pr view $ f0cal hw add tx2 --query ‘NVidia TX2’ $ f0cal pr view --hw=tx2 $ f0cal pr view --hw=my_laptop --hw=tx2 $ f0cal pr view --hw=my_laptop --hw=tx2 --gui
- 10. $ curl -o https://bootstrap.f0cal.com -L bootstrap-f0cal $ python3 bootstrap-f0cal git -v _venv37 $ source _venv37/bin/activate $ f0cal hw add my_laptop --detect $ f0cal sw install opencv/4.0.1 $ git clone https://github.com/learnopencv/learnopencv.git $ cd learnopencv/MultiObjectTracker $ f0cal sw build ./ $ f0cal pr add -- ./bin/multitracker $ f0cal pr view $ f0cal hw add tx2 --query ‘NVidia TX2’ $ f0cal pr view --hw=tx2 $ f0cal pr view --hw=my_laptop --hw=tx2 $ f0cal pr view --hw=my_laptop --hw=tx2 --gui hello_world time
- 11. $ curl -o https://bootstrap.f0cal.com -L bootstrap-f0cal $ python3 bootstrap-f0cal git -v _venv37 $ source _venv37/bin/activate $ f0cal hw add my_laptop --detect $ f0cal sw install opencv/4.0.1 $ git clone https://github.com/learnopencv/learnopencv.git $ cd learnopencv/MultiObjectTracker $ f0cal sw build ./ $ f0cal pr add -- ./bin/multitracker $ f0cal pr view $ f0cal hw add tx2 --query ‘NVidia TX2’ $ f0cal pr view --hw=tx2 $ f0cal pr view --hw=my_laptop --hw=tx2 $ f0cal pr view --hw=my_laptop --hw=tx2 --gui Hardware/ software co-design
- 12. $ curl -o https://bootstrap.f0cal.com -L bootstrap-f0cal $ python3 bootstrap-f0cal git -v _venv37 $ source _venv37/bin/activate $ f0cal hw add my_laptop --detect $ f0cal sw install opencv/4.0.1 $ git clone https://github.com/learnopencv/learnopencv.git $ cd learnopencv/MultiObjectTracker $ f0cal sw build ./ $ f0cal pr add -- ./bin/multitracker $ f0cal pr view $ f0cal hw add tx2 --query ‘NVidia TX2’ $ f0cal pr view --hw=tx2 $ f0cal pr view --hw=my_laptop --hw=tx2 $ f0cal pr view --hw=my_laptop --hw=tx2 --gui
- 13. $ curl -o https://bootstrap.f0cal.com -L bootstrap-f0cal $ python3 bootstrap-f0cal git -v _venv37 $ source _venv37/bin/activate $ f0cal hw add my_laptop --detect $ f0cal sw install opencv/4.0.1 $ git clone https://github.com/learnopencv/learnopencv.git $ cd learnopencv/MultiObjectTracker $ f0cal sw build ./ $ f0cal pr add -- ./bin/multitracker $ f0cal pr view $ f0cal hw add tx2 --query ‘NVidia TX2’ $ f0cal pr view --hw=tx2 $ f0cal pr view --hw=my_laptop --hw=tx2 $ f0cal pr view --hw=my_laptop --hw=tx2 --gui Sophisticated OS hacking
- 14. my_laptop ====================================================================================================== Latency (ms) Throughput (fps) count min max mean max min mean prefix width height cv::add 640 480 1158 0.02 0.09 0.05 48461.35 10861.18 19247.80 cv::addWeighted 640 480 772 0.20 1.09 0.47 5059.68 919.17 2142.63 cv::bitwise_and 640 480 2316 0.29 1.42 0.74 3456.45 704.70 1354.75 cv::bitwise_not 640 480 772 0.01 0.18 0.04 69132.39 5417.38 25808.52 cv::countNonZero 3 3 1158 0.00 0.05 0.00 1221001.22 18233.54 302150.20 cv::cvtColor 640 480 2050 0.03 8.16 1.16 33158.70 122.55 859.75 cv::dilate 3 3 772 0.06 0.38 0.17 17585.82 2637.09 5760.23 640 480 1544 0.06 0.38 0.17 17585.82 2637.09 5760.23 cv::erode 3 3 386 0.07 0.34 0.19 14511.05 2926.26 5187.10 640 480 772 0.07 0.34 0.19 14511.05 2926.26 5187.10 cv::inRange 1 4 1544 0.25 1.17 0.80 3983.87 852.97 1253.34 640 480 772 0.25 1.17 0.80 3983.87 852.97 1253.34 cv::morphologyEx 3 3 772 0.06 0.74 0.28 16902.17 1343.41 3574.38 640 480 1544 0.06 0.74 0.28 16902.17 1343.41 3574.38
- 16. $ curl -o https://bootstrap.f0cal.com -L bootstrap-f0cal $ python3 bootstrap-f0cal git -v _venv37 $ source _venv37/bin/activate $ f0cal hw add my_laptop --detect $ f0cal sw install opencv/4.0.1 $ git clone https://github.com/learnopencv/learnopencv.git $ cd learnopencv/MultiObjectTracker $ f0cal sw build ./ $ f0cal pr add -- ./bin/multitracker $ f0cal pr view $ f0cal hw add tx2 --query ‘NVidia TX2’ $ f0cal pr view --hw=tx2 $ f0cal pr view --hw=my_laptop --hw=tx2 $ f0cal pr view --hw=my_laptop --hw=tx2 --gui Value.
- 17. tx2 ====================================================================================================== Latency (ms) Throughput (fps) count min max mean max min mean prefix width height cv::add 640 480 1158 0.02 0.09 0.05 48461.35 10861.18 19247.80 cv::addWeighted 640 480 772 0.20 1.09 0.47 5059.68 919.17 2142.63 cv::bitwise_and 640 480 2316 0.29 1.42 0.74 3456.45 704.70 1354.75 cv::bitwise_not 640 480 772 0.01 0.18 0.04 69132.39 5417.38 25808.52 cv::countNonZero 3 3 1158 0.00 0.05 0.00 1221001.22 18233.54 302150.20 cv::cvtColor 640 480 2050 0.03 8.16 1.16 33158.70 122.55 859.75 cv::dilate 3 3 772 0.06 0.38 0.17 17585.82 2637.09 5760.23 640 480 1544 0.06 0.38 0.17 17585.82 2637.09 5760.23 cv::erode 3 3 386 0.07 0.34 0.19 14511.05 2926.26 5187.10 640 480 772 0.07 0.34 0.19 14511.05 2926.26 5187.10 cv::inRange 1 4 1544 0.25 1.17 0.80 3983.87 852.97 1253.34 640 480 772 0.25 1.17 0.80 3983.87 852.97 1253.34 cv::morphologyEx 3 3 772 0.06 0.74 0.28 16902.17 1343.41 3574.38 640 480 1544 0.06 0.74 0.28 16902.17 1343.41 3574.38
- 18. $ curl -o https://bootstrap.f0cal.com -L bootstrap-f0cal $ python3 bootstrap-f0cal git -v _venv37 $ source _venv37/bin/activate $ f0cal hw add my_laptop --detect $ f0cal sw install opencv/4.0.1 $ git clone https://github.com/learnopencv/learnopencv.git $ cd learnopencv/MultiObjectTracker $ f0cal sw build ./ $ f0cal pr add -- ./bin/multitracker $ f0cal pr view $ f0cal hw add tx2 --query ‘NVidia TX2’ $ f0cal pr view --hw=tx2 $ f0cal pr view --hw=my_laptop --hw=tx2 $ f0cal pr view --hw=my_laptop --hw=tx2 --gui
- 19. $ curl -o https://bootstrap.f0cal.com -L bootstrap-f0cal $ python3 bootstrap-f0cal git -v _venv37 $ source _venv37/bin/activate $ f0cal hw add my_laptop --detect $ f0cal sw install opencv/4.0.1 $ git clone https://github.com/learnopencv/learnopencv.git $ cd learnopencv/MultiObjectTracker $ f0cal sw build ./ $ f0cal pr add -- ./bin/multitracker $ f0cal pr view $ f0cal hw add tx2 --query ‘NVidia TX2’ $ f0cal pr view --hw=tx2 $ f0cal pr view --hw=my_laptop --hw=tx2 $ f0cal pr view --hw=my_laptop --hw=tx2 --gui Beauty.
- 21. Design-Partner Program https://www.f0cal.com/early-access-request 3 months of glory Work closely with FØCAL engineers Contribute to product roadmap jr@f0cal.com br@f0cal.com