Downloaded 33 times
Uploaded byMarcos Nieto
Optimization of computer vision algorithms in codesign methodologies
The document discusses the optimization of computer vision algorithms for advanced driver assistance systems (ADAS) using co-design methodologies. It details various vehicle detection prototypes, addressing challenges and solutions that improved performance, leading to a tenfold speed-up without extensive code optimization. The conclusions highlight the benefits of parallel hardware and software development and iterative redesign for reducing time-to-market for computer vision systems.
Optimization of computer vision algorithms in codesign methodologies
- 1. Optimization of computervision algorithms in codesign methodologies Marcos Nieto, Juan Diego Ortega, Oihana Otaegui, Andoni Cortés Vicomtech-IK4 September 8, 2014 1 1 - ITS World Congress 2014, September 8 -
- 2. Overview • Motivation- ADAS • Computer vision for ADAS • Codesign Methodology • Test case – Vehicle detection Prototype #1 – Vehicle detection Prototype #2 – Vehicle detection Prototype #3 • Results • Conclusions September 8, 2014 2 1
- 3. Motivation - ADAS • Advanced Driver Assistance Systems (ADAS) ↑ Safety ↑ Traffic Mobility ↓ Energy Consumption ADAS Advanced Driver Assistance Systems September 8, 2014 3 2
- 4. Computer vision forADAS • Camera sensors – Cheap electronics – Greater quantity and quality of information – Perception similar to human – Can be fused with other sensors/systems: GPS, RADAR, V2V, V2I Traffic sign recognition Safety distance Driver Monitoring Lane Keeping Pedestrian detection September 8, 2014 4 3
- 5. Computer vision forADAS • Embedding computer vision algorithms is a challenge – (HW) Specify (camera, illumination, algorithm...) – (SW) Optimize algorithm – (HW) Select target processor – (SW/HW) Fine tune algorithm parameters – (SW/HW) Optimize code • Fit process flow to HW capabilities • Exploit massive parallelization HW C++ C, Assembly (ARM) C++ VHDL, CUDA (FPGA, GPU) – (SW/HW) Add installation / calibration / maintenance / data logging procedures September 8, 2014 5 4
- 6. Codesign Methodology •Coordinating SW and HW developments – Using Re-configurable HW platforms – Exploiting platform-independent SW libraries Co-design development cycle Traditional SW/HW development cycle User requirements Design SW Design/Select HW Integrate SW in HW - Prototype Profile SW in HW Protpotype Validation September 8, 2014 6 User requirements Design SW Design/Select HW Integrate SW in HW - Prototype Profile SW in HW Protpotype Validation Iterate Last iteration Iterate Iterate 5
- 7. Codesign Methodology •SW and HW developments can run in parallel – More efficient joint development – SW design can be revised after profiling reports in target HW – Expensive code migration/optimization is significantly minimized “It is better to revise the algorithm rather than spend months optimizing the code” • The development cycle – First SW prototype can be directly tested in HW – Profiling provides bottleneck information – Revision of SW design solves detected bottleneck – Iterate until SW and HW meet requirements – Optimize final SW in target HW September 8, 2014 7 6
- 8. Vehicle detection Prototype#1 • Multiscale scanning and detection-by-classification – Traditional (and successful) approach in computer vision – A car model is trained with a large dataset of example images – New images are scanned at different scales and each candidate region is Multiscale HOG compared with the model September 8, 2014 8 Adaboost Clustering SVM 1.0 Tracking Shadows, edges, symmetry Training Training Images Brute-force Multiscale Scanning 7
- 9. Vehicle detection Prototype#2 • Bottleneck #1: – It can propose absurd hypotheses according to size and position that are anyway compared with the model • Solution: – Exploit perspective information to create a grid of possible locations of vehicles September 8, 2014 9 Training Training Perspective Adaboost Clustering SVM 2.0 Tracking HOG Shadows, edges, symmetry Images 8
- 10. Vehicle detection Prototype#3 • Bottleneck #3: – Some hypotheses are really likely not cars, and the comparison with the model is costly • Solution: – No need to fully compare with the trained model – Run pre-analysis to discard clearly not car candidates using simpler features: shadows, simmetry, edges, etc. September 8, 2014 10 Perspective Training Pre-analysis / Integral SVM 2.0 Tracking Perspective clustering HOG Shadows, edges, symmetry Images Pre-selected candidates 9
- 11. Results • Profilecomparison Xilinx Zynq Dual ARM® Cortex-A9 MPCore, 866 MHz, 1 GB – Only CPU, without code optimization – 10x speed up from prototype #1 to #3 -> real time performance 450 400 350 300 250 200 150 100 50 0 September 8, 2014 11 10 Features (ms) HOG (ms) Adaboost Cluster SVM (ms) Av. Time (ms) Prototype #1 135.88 252.23 18.45 0 13.67 420.23 Prototype #2 56.43 112.56 5.88 0 4.12 178.99 Prototype #3 11.23 6.19 0 10.22 4.12 31.76 Time (ms) per frame
- 12. Results • Samplevideo(s) PETS 2001 dataset September 8, 2014 12 11 Frontal camera Rear camera
- 13. Conclusions • Co-designstrategies can reduce time-to-market for computer vision systems • It allows SW and HW teams work in parallel • Iterative SW re-design is possible • We have shown a vehicle detection example • Three prototypes with x10 speed-up without code optimization nor use of special HW September 8, 2014 13 12
- 14. Live Demo atour booth: Booth 3008 Thank You! September 8, 2014 14 13
Editor's Notes
- #4 -ADAS will increase driving safety, enhance traffic mobility and optimize energy consumption worldwide.