Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

"Distance Estimation Solutions for ADAS and Automated Driving," a Presentation from AImotive

34 views

Published on

For the full video of this presentation, please visit:
https://www.embedded-vision.com/platinum-members/aimotive/embedded-vision-training/videos/pages/may-2019-embedded-vision-summit-debreczeni

For more information about embedded vision, please visit:
http://www.embedded-vision.com

Gergely Debreczeni, Chief Scientist at AImotive, presents the "Distance Estimation Solutions for ADAS and Automated Driving" tutorial at the May 2019 Embedded Vision Summit.

Distance estimation is at the heart of automotive driver assistance systems (ADAS) and automated driving (AD). Simply stated, safe operation of vehicles requires robust distance estimation. Many different types of sensors (camera, radar, LiDAR, sonar) can be used for distance estimation, and different distance estimation techniques can be used with each type of sensor. Each type of sensor and technique has unique strengths and weaknesses. Debreczeni examines these techniques and their strengths and weaknesses, and shows how multiple techniques using different sensor types can be fused to enable robust distance estimation for a specific automated driving application.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

"Distance Estimation Solutions for ADAS and Automated Driving," a Presentation from AImotive

  1. 1. © 2019 AImotive Distance Estimation Solutions for ADAS and Automated Driving Gergely Debreczeni Aimotive May 2019
  2. 2. © 2019 AImotive Introduction On the importance of distance estimation Vehicle automation has many purposes but they all share a common goal: they must be safe! Being safe means – among others – to not hit anybody or anything and for this we need to know the distance of objects. Realizing this goal requires distance estimation methods: • that are precise and robust • based on redundant and diverse hardware and software solutions • using sensors with different working principles • using the least amount of a-priori assumption 2
  3. 3. © 2019 AImotive Distance estimation with radars 3
  4. 4. © 2019 AImotive Radars Distance estimation with radars • Radars – used in ADAS – have been developed (and still being developed) for decades. • Early flash radars have already been replaced by FMCW (Frequency Modulated Continuous Wave) solution. 4 • The FMCW operation principle: • Transmits FM electromagnetic waves • Receives the backscattered waves • Compares timing and frequency of outgoing and incoming waves • Calculates speed and range • Advantages • Direct physical measurement of speed and distance with high resolution • Works in adversarial weather conditions • Range independent precision • Disadvantages • Lower angular resolution • Material with low reflectivity may not be detected
  5. 5. © 2019 AImotive Distance estimation with LIDARs 5
  6. 6. © 2019 AImotive LIDARs Distance estimation with T.O.F LIDARs • LIDARs (Light Detection and Ranging Device) have different operation principles: • TOF – Time of Flight principle • Flash – Integrated light intensity • FMCW – like FMCW radars Distance estimation with TOF LIDARs: • Laser impulse is emitted • Returning signal’s timing is measured • Divided with the speed of light. 6 • Advantages • Simple operation principle • Simple implementation • Range independent precision • Disadvantages • Problems at very close range • First obstacle reflections could cause problems • Requires far greater outgoing laser energy than modulated solutions • Rain, snow, etc. can cause false alarms
  7. 7. © 2019 AImotive LIDARs Distance estimation with T.O.F LIDARs 7
  8. 8. © 2019 AImotive LIDARs Distance estimation with flash LIDARs 8
  9. 9. © 2019 AImotive LIDARs Distance estimation with flash LIDARs 9 Distance estimation with flash LIDARs: • Reference image • Laser pulse is emitted • Intensity measurement sensor exposition is synchronized with laser emission • Measurement image • Laser pulse is emitted • Exposition is time delayed • Integrated light intensity is proportional to distance • Advantages • More robust to environmental conditions • Global shutter operation results in real snapshots • No moving parts, no scanning • Excellent angular resolution • Disadvantages • Long range performance is questionable • Requires high energy for scaling • Resolution is limited by shot noise
  10. 10. © 2019 AImotive Distance estimation with ultrasonic sensors 10
  11. 11. © 2019 AImotive Ultrasonic sensors Distance estimation with ultrasonic sensors Operation principle of ultrasonic • Based on propagation of sound waves • Same time-of-flight principle 11 • Advantages • Extremely simple • Cheap • Works in many diverse conditions • Disadvantages • Because of slow speed of sound FPS rate is limited • Only for proximity sensing (up-to c.c. 12 meters at max) • Practically no angular resolution • Reliability decreases with increasing ego- vehicle speed • Different reflectivity materials could cause misdetections
  12. 12. © 2019 AImotive Distance estimation with cameras 12
  13. 13. © 2019 AImotive Cameras Distance estimation with stereo camera setup Typical stereo solutions • Arrangements • transverse stereo setup • longitudinal stereo setup • Feature matching • Classical • AI-based 13 Advantages • Simple principle • Partially relies on physical information, i.e. using triangulation • Very high angular resolution • Extremely cheap Disadvantages • Most of the solution sensitive to calibration errors • Feature matching could introduce errors, especially in featureless regions • Distance resolution depends on the distance
  14. 14. © 2019 AImotive Cameras Distance estimation with AI and mono camera 1. Single camera, single frame AI based distance estimation 2. AI based from stereo camera image 14 • Advantages • Can be trained in supervised and in unsupervised manner (1,2) • Different forms can interpret higher level of semantic information, increasing precision (1,2) • Works on a single camera, as well (1) • Disadvantages • Precision must be carefully validated on statistically representative data (1) • Same as in classical stereo (2)
  15. 15. © 2019 AImotive Cameras Distance estimation using known size objects Method • Knowing the • physical size of an object • focal length of the camera one can easily calculate the distance of the object by • Measuring its screen space size and • using “similar triangles”. • The method corresponds to a virtual stereo setup having a base distance equivalent to that of the object size. 15 Advantages • Can be very precise since object size corresponds to virtual stereo rig size Disadvantages • Requires precise classification of objects with known size and a-priori knowledge of object size • Same as for “normal” stereo setup
  16. 16. © 2019 AImotive Cameras Distance estimation using object proportionality 16 Method • Objects entering and exiting from ego-vehicle sensor-range. • Object size can be precisely measured (for example from stereo vision) when they are close- by. • When those objects move away and stereo doesn’t see them any more, “known size object” method can be used. Disadvantages • Requires precise tracking • Works only for objects moving away and not for objects entering the sensor range Advantages • Can be very precise • Requires no assumption • Can be used with mono and stereo camera
  17. 17. © 2019 AImotive Cameras Distance estimation using ground plane estimation Method • Knowing the • Height and orientation of the camera • The road surface geometry one can easily calculate the distance of the object by • determining the road / object contact point in screen space and • using “similar triangles”. • The method corresponds to a virtual stereo setup having a base distance equivalent to that of the camera height 17 Advantages • Can be very precise since camera height corresponds to virtual stereo rig size • Good approximation for close-by objects (ground plane assumption) Disadvantages • Requires precise contact point defemination • Very sensitive to camera pitch estimation errors especially at long distances
  18. 18. © 2019 AImotive Cameras Distance (TTC) estimation based on scale change Scale change, TTC • Visual scale change of an object (because it is moving away or approaching the camera) can directly be translated to TTC (Time To Collision) • Uses same principle as SFM or SLAM but absolute scale end ego-motion is not necessary. 18 • Advantages • Scale change can be measured at a very high level (10th of a pixel precision) • No need for calibration • Also works on mono camera • In contrast to stereo smaller feature matching scan distances are required • Disadvantages • No absolute scale • Error goes at third power of distance, only good for small distances, i.e. up-to 2-3 seconds of TTC • Needs non-negligible relative motion
  19. 19. © 2019 AImotive Cameras Distance estimation using SFM / SLAM SFM and SLAM • Both uses geometric triangulation • Based on feature matching • Iterative optimization of 3D point position and camera position, orientation 19 Advantages • Works for mono and for stereo camera • The framework is flexible, many feature matching and optimization algorithms can be used Disadvantages • Prone to feature matching or ego-motion errors • Mono camera solution requires motion parallax
  20. 20. © 2019 AImotive Cameras Distance estimation based on light intensity - passive Method • The intensity of light decreases quadratically as a function of distance • Measuring that intensity profile at two distinct points one can determine distance of the light source 20 Advantages • Only passive measurement which purely relies on the laws of physics • No a-priori assumption has to be made • Simple principle, simple implementation • Works perfectly for full integrable point- like light source • Dynamical parameters can be adjusted on-the-fly Disadvantages • Precision is limited by the object segmentation precision • Objects lying under different angles from the camera can have different reflectivity profile
  21. 21. © 2019 AImotive Cameras Distance estimation based on light intensity - active The method 1. two light sources with different attenuation profile 1. for example a point-like source (1/r2) 2. And a linear light source (1/r) 2. a camera 3. take an image with light source #1 on 4. take an image with light source #2 on 5. the ratio of the two intensity image can be used to determine distance 21 Disadvantages • Requires two (calibrated) light source • Optimal performance is guaranteed only in a small segment of FOV. • Good for proximity sensing only Advantages • Relies on laws of physics • Can be very good proximity sensing solution
  22. 22. © 2019 AImotive Cameras Comparison of the methods 22 • Different methods have different accuracy and different distance dependence • Meaningful fusion algorithm must take into account these properties • Realistic fusion algorithms must handle association and false alarm errors as well.
  23. 23. © 2019 AImotive Cameras How to fuse, when to fuse ? 23 An example for distance information fusion with stereo cameras • Use pixel wise stereo for close by generic obstacle detection and distance estimation • Use AI base known object size and object proportionality for distances larger then stereo range. • Compensate bounding box uncertainty with sparse optical flow based scale change measurements An example for distance information fusion with mono camera • Use ground plane estimation for close- by objects. • Use lane based distance estimation with AI based known size objects for obstacles further away. • Use AI based free space estimation or mono camera SLAM for drivable free space estimation There is no generic recipe for fusion algorithm, it heavily depends on the sensor setup, operation domain and on the capabilities and false alarm rates of available methods.
  24. 24. © 2019 AImotive Summary and conclusion 24
  25. 25. © 2019 AImotive Summary Some concluding remarks • Distance estimation is probably one of the most important ADAS and AD features • It must be right! • There are more than 15 different methods to achieve it: • active / passive sensors • classical algorithm / laws of physics / learning based • using assumptions / assumption free • correlated / non – correlated • With clever combination of all the above an extremely safe system can be developed. 25
  26. 26. © 2019 AImotive Resources 26 • AIMOTIVE • https://www.aimotive.com • FMCW radar • https://en.wikipedia.org/wiki/Continuous- wave_radar • AI mono depth • https://arxiv.org/abs/1901.09402 • https://arxiv.org/abs/1812.11941 • https://arxiv.org/abs/1903.03273 • Stereo depth • http://citeseerx.ist.psu.edu/viewdoc/downloa d?doi=10.1.1.88.8897&rep=rep1&type=pdf • https://arxiv.org/pdf/1610.04121.pdf • AI stereo • https://arxiv.org/abs/1904.02957 • https://arxiv.org/abs/1709.00930 Embedded Vision Summit “Distance estimation in ADAS and AD” (2019, 24, May)

×