Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Building ADAS
SYSTEM from scratch
Alex Myakov, Chief CV Advocate
Yury Gorbachev, CV Integration & Optimization lead
Septem...
Who we are ?
Itseez was acquired by IoTG/Intel in July, 2016
Itseez was founded in 2005:
• 3 ex- Intel co-Founders + 1 Pri...
Building ADAS System from Scratch
The Dream and Ambition:
• Create state-of-the-art software based front
camera ADAS algos...
Building ADAS solution from scratch
Strategy:
• Highly portable CV algos (pure ARM optimized code)
• Open SW platform (And...
ADAS Project Timeline
2013
• TSR
• Demo platform:
Nexus 4
2014
• +LDW
• +FCW
• Demo Platform:
Asus Transformer
Tablet + An...
Demo Setup
Camera:
• 1M
• 1280H x 800V
• HDR/WDR
• 30 fps
Embedded
Platform
USB 3.0
Snap-and-go concept:
• Simple and fast...
Lessons Learned
We expected ADAS to be just another CV application !
• we ended up running into and solving lots of issues...
ADAS Solution SW Architecture
TSR LDW FCW PD
Common Image Processing Pipeline + Autocalibration
OpenCV
IPP AcceleratedCV (...
SW design approach
ADAS algorithms are purely software based:
• Possible to design and test on desktops
• Purely based on ...
Continuous integration
Any change in ADAS algorithms or processing
pipeline requires complete re-evaluation
• Detection/pr...
Datasets
CV algorithms require datasets for design and
testing:
• No available commercial datasets
• Research datasets can...
Datasets stats
TSR: 2.5K good unique signs
PD: 83K+ pedestrian bounding boxes
FCW: 5K+ different cars and ca 1K trucks
LDW...
HW Issues
HW issues are caused by consumer “gradeness” of
components
Temperature issues:
• Camera overheating -> skipped o...
Where is Deep Learning in our algos?
Original ADAS algos were based on classical CV
• Embedded platforms were too weak and...
Semantic Road Segmentation - PoC
Obstacle Detection using SfM - PoC
•We estimate 3D coordinates using
points tracking and vehicle speed.
•Obstacles are cal...
Semantic segmentation + Obstacle Detection
Deep Learning - Silver Bullet ?
• Great new CV tool
• Large datasets
• A few orders of magnitude more compute
than classic...
Questions ?
Contact Details:
Alex Myakov, Chief CV Advocate
Email: alex.myakov@intel.com
You’ve finished this document.
Download and read it offline.
Upcoming SlideShare
The flex ray protocol
Next
Upcoming SlideShare
The flex ray protocol
Next
Download to read offline and view in fullscreen.

1

Share

Building ADAS system from scratch

Download to read offline

Presentation at Autosens 2016

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Building ADAS system from scratch

  1. 1. Building ADAS SYSTEM from scratch Alex Myakov, Chief CV Advocate Yury Gorbachev, CV Integration & Optimization lead September, 2016
  2. 2. Who we are ? Itseez was acquired by IoTG/Intel in July, 2016 Itseez was founded in 2005: • 3 ex- Intel co-Founders + 1 Principal Engineer • OpenCV development and support (2005-present) • OpenVX initiative leaders: v1.0 and v1.1 were published in October, 2014 and in May, 2016 SW Products: ADAS, Facense, AcceleratedCV (ACV) Skills: CV algorithms, HW specific optimization, 3+ years of deep learning (DL) Industries: automotive, security, robotics, wearables,etc
  3. 3. Building ADAS System from Scratch The Dream and Ambition: • Create state-of-the-art software based front camera ADAS algos • License such algos to Tier-1s and OEMs Starting point (late 2013): • Strong knowledge of CV • Strong knowledge of embedded/optimization • Good knowledge of cameras/sensors/optics • No ADAS/automotive specific knowledge
  4. 4. Building ADAS solution from scratch Strategy: • Highly portable CV algos (pure ARM optimized code) • Open SW platform (Android, Linux) • Simplest system possible • Cheapest COTS components • Camera (optics, sensor, packaging) • HW platform • Easy/fast installation in any car with no dependence on car parameters • Automatic or simple calibration
  5. 5. ADAS Project Timeline 2013 • TSR • Demo platform: Nexus 4 2014 • +LDW • +FCW • Demo Platform: Asus Transformer Tablet + Android OS + Standalone Camera 2015 • +PD • Demo Platform: TK1+ Linux OS + Standalone Camera • Demos with QNX: • CES 2015 • TU Update 2015 2016 • PoCs: • +Semantic Road Segmentation • +Obstacle Detection • +Driver Monitoring • Demos with QNX: • CES 2016
  6. 6. Demo Setup Camera: • 1M • 1280H x 800V • HDR/WDR • 30 fps Embedded Platform USB 3.0 Snap-and-go concept: • Simple and fast installation in any car • No dependence on car parameters • Automatic calibration or simple calibration
  7. 7. Lessons Learned We expected ADAS to be just another CV application ! • we ended up running into and solving lots of issues • SW development/testing paradigm • HW issues • Datasets
  8. 8. ADAS Solution SW Architecture TSR LDW FCW PD Common Image Processing Pipeline + Autocalibration OpenCV IPP AcceleratedCV (ACV) x86 ARM CV algo prototyping on desktop Lab testing/CI on server Execution on target Live test/benchmarking
  9. 9. SW design approach ADAS algorithms are purely software based: • Possible to design and test on desktops • Purely based on OpenCV • No special software skills are required (GPU, DSP, etc) • Flexible and upgradable Solved platform compatibility issues • No vendor provides cross-platform CV framework • OpenCV is limited in supporting this Created AcceleratedCV (ACV) library to address platform compatibility issue
  10. 10. Continuous integration Any change in ADAS algorithms or processing pipeline requires complete re-evaluation • Detection/processing quality on the entire dataset • Performance figures for each ADAS algorithm Benefits from pure SW based approach • Quality evaluation on servers/cloud for entire dataset • Performance benchmarking on multiple HW targets • Reduces test time from days to hours!
  11. 11. Datasets CV algorithms require datasets for design and testing: • No available commercial datasets • Research datasets cannot be used for products Our own datasets for each ADAS algo were created: • Different conditions (rain, snow, sun) • Geographical locations • Dataset annotation and management tools • Many days of driving + many months of annotation The market offering of quality annotated datasets is still very limited !
  12. 12. Datasets stats TSR: 2.5K good unique signs PD: 83K+ pedestrian bounding boxes FCW: 5K+ different cars and ca 1K trucks LDW: 0.5M+ boundaries
  13. 13. HW Issues HW issues are caused by consumer “gradeness” of components Temperature issues: • Camera overheating -> skipped or corrupted frames • HW platform overheating -> throttling System issues: • throttling and unpredicted system behavior under heavy processing loads Mechanical issues: • USB 3.0 cable connectors get loose and break
  14. 14. Where is Deep Learning in our algos? Original ADAS algos were based on classical CV • Embedded platforms were too weak and not able to provide required performance Gflops • Datasets were too small to yield quality DL results • DL technology was not fully there Conventional CV + small DL networks: • PD: validation of PD results based on classical CV – increase DR and reduce FA rate • Driver monitoring: conventional face detector + DL based headpose estimation
  15. 15. Semantic Road Segmentation - PoC
  16. 16. Obstacle Detection using SfM - PoC •We estimate 3D coordinates using points tracking and vehicle speed. •Obstacles are calculated as clusters of points above the road plane.
  17. 17. Semantic segmentation + Obstacle Detection
  18. 18. Deep Learning - Silver Bullet ? • Great new CV tool • Large datasets • A few orders of magnitude more compute than classic CV • FuSa implications
  19. 19. Questions ? Contact Details: Alex Myakov, Chief CV Advocate Email: alex.myakov@intel.com
  • vhsu

    Jun. 20, 2017

Presentation at Autosens 2016

Views

Total views

1,848

On Slideshare

0

From embeds

0

Number of embeds

9

Actions

Downloads

75

Shares

0

Comments

0

Likes

1

×