Mercedes - Autonomous Driving - The S500 Intelligent drive


Published on

Presentation of Ralf Herrtwich about autonomous driving and the approach to this by Mercedes Benz

Published in: Automotive, Business, Education

Mercedes - Autonomous Driving - The S500 Intelligent drive

  1. 1. Autonomous Driving to Advance Road Safety and Comfort The S500 INTELLIGENT DRIVE Ralf Herrtwich Driver Assistance and Chassis Systems - Group Research and Advanced Engineering
  2. 2. Project Goals Highly autonomous driving on the original Bertha Benz Route from Mannheim to Pforzheim 125 years after the world's first overland drive with as few driver interventions as possible • floating within regular traffic • with no special guidance in front • with no alterations of given infrastructure • Goals: Gain experience with autonomous driving beyond highways • Experiment with a vehicle setup with regular production or close-to-production sensors (only more) • Cameras (mono and stereo) • Radars • Identify technical issues and examine different solutions for future autonomous functions • Prepare for innovation beyond 2020 • Push the envelope •
  3. 3. Why We Wanted to Do It
  4. 4. Mercedes-Benz Intelligent Drive Accident-free driving Autonomous driving Prevent and avoid accidents altogether or at least mitigate their effects to an amount of minimum harm and damage Assist drivers with maneuvers if they want it, but only when and where it is technically possible without taking imponderable risks Example: PRE-SAFE BRAKE Example: DISTRONIC PLUS Safety Comfort Key to the Mercedes-Benz Brand
  5. 5. Autonomous Driving in the New S- and E-Class Mercedes-Benz Intelligent Drive Stop & Go Pilot: Vehicle moves autonomously at low speeds in traffic jams • Longitudinal and lateral control incl. lane keeping in curves • Lane mark detection • Observation of vehicle ahead (swarm mode) • Intelligent hands-on detection • Driver is called back into the loop when needed • Every 10-15 seconds as soon as vehicle drives 30 km/h or more •
  6. 6. Autonomous? αὐτο "self" + νόμος "law" Autonomy (as seen by Immanuel Kant): The capacity of an individual or entity to make an informed, un-coerced decision within principles of reason. Observations Rules of behaviour Actions
  7. 7. Stages of Autonomy (NHTSA Logic) To here: From here: Level 0 No automation Driver in charge Level 1 Function-specific automation Single control functions such as speed selection, braking or lane keeping are automated Level 2 Combined function automation More than one control function is automated Driver expected to be available for control at all times and on short notice Level 3 Limited self-driving automation Vehicle takes control most of the time Driver expected to be available for occasional control with comfortable transition times Level 4 Full self-driving automation Vehicle takes control all of the time Driver not expected to be available for control at any time
  8. 8. Complexity of Automation Low ego velocity High ego velocity Structured traffic environment Traffic Jam Highway Step1 Step 3 Chaotic traffic environment Parking Off-Highway Step 2 Bertha Benz Step 4 Drive
  9. 9. Complexity of Automation Highway Off-highway Lanes Wide Narrow and not necessarily exclusive Direction Same direction for all Vehicles from and in all directions • Maneuvers Lane Change • • Passing of in-road obstacles (e.g. parked cars) incl. coordination with oncoming traffic Intersection navigation incl. turning Roundabouts Traffic signs Speed limits Traffic lights and priority rules Road users Vehicles only Pedestrians and cyclists all around Field of view Wide Obstructed
  10. 10. Where We Did It
  11. 11. The Historic Bertha Benz Route In August 1888, Bertha Benz took her husband's Patentmotorwagen at dawn and, together with her two sons, drove to Pforzheim to visit her family. She arrived shortly before midnight after what later became known as the world's first overland drive.
  12. 12. Route Difficulty (Original Estimation) Pforzheim Mannheim N
  13. 13. Route Difficulty (Actual Difficulty) Pforzheim Heidelberg Passage Nussloch Passage Pfinztal Passage Ladenburg Passage Bruchsal Passage Mannheim Passage Mannheim Weingarten Passage N
  14. 14. How We Did It
  15. 15. Vehicle Base Regular S 500 with all emergency braking systems enabled as underlying protection
  16. 16. Sensors
  17. 17. Actuators Steering Acceleration / Braking Special software version for Electrical Power Steering to allow for larger steering angles at higher speeds Special software version for DISTRONIC to limit initial acceleration and braking deceleration (212 4x4 ZF EPS) (222 MB DISTRONIC)
  18. 18. System Structure Sensors Artificial Intelligence Actuators Feature Map Feature Loc Camera ESP Planning Map Feature Localization Localization Vehicle Loc Sensors Planning EPS Lane Map Lane Localization Stereo Camera RDU Emulation Object Detection Traffic Light Map Traffic Light Camera Traffic Light Detection 360° Radars Radar Processing DISTRONIC Radars ESP Sensor Cluster Visualization
  19. 19. What the Car Sees
  20. 20. Stereo Vision Left Image Disparity (Distance) Image Right Image Color encoded distance: close ….. far
  21. 21. Object Recognition: Pedestrians Input Hypothesis Classification Pedestrian examples Tracking Non-pedestrian examples Image Depth
  22. 22. Traffic Light Recognition Traffic light recognition is not as easy as one may think since When stopping in front of the traffic light, it must be in the field of view • When approaching a traffic light on rural roads, it must be visible at large distances • At intersections, we have to find “our” traffic light •
  23. 23. Traffic Light Recognition Easy Medium Hard Impossible
  24. 24. What the Car Knows
  25. 25. What's in a Map? Vehicle Stop line Traffic light Trajectory plan: color = confidence
  26. 26. Map Architecture Different map layers serve different purposes. Navi map (GDF) Planning layer Localization layers
  27. 27. Map Accuracy Speed limit +/- 10 m Relative accuracy of map features of 10 cm desirable Curb +/- 5 cm Map data Lane marking +/- 5 cm Traffic light +/- 10 cm Camera image of road surface
  28. 28. Vehicle Localization [ [ [ 1. Landmarks for localization are defined. ] ] ] 2. A map is created with such localization landmarks. 3. A camera detects landmarks. Those are matched with landmark maps to compute the actual pose of the vehicle.
  29. 29. Landmark Robustness and Diversity Different environments require different types of landmarks. Rural: "Lane Loc" Curbs Lane Markings Urban: "Feature Loc" Edge Detector Collaboration with KIT
  30. 30. Map Generation To explore map generation for autonomous driving, Daimler has entered into a cooperation with Nokia HERE as one of the world's leading map suppliers. Exploration areas: Content needs (and accuracy needs) for autonomous driving • Localization support via landmarks in maps • Scalable and reliable data capturing methods • Online map updating • Online map insufficiency reporting incl. crowd sourcing concepts • "Path clearance" concepts • Source: HERE
  31. 31. What the Car Thinks
  32. 32. Maneuvering Tasks Behaviour generation Object classification and prediction Trajectory planning
  33. 33. Object Classification and Prediction Hypothesis generation for movements of dynamic objects based on Prediction Path A (unlikely) Object attributes • Driving lanes • Probabilistic evaluation of hypotheses Prediction w/o lane information Prediction Path B (likely)
  34. 34. Behaviour Generation Hierarchical concurrent state machine to determine vehicle behaviour in individual situations Collaboration with KIT (former AnnieWay technology from Urban Challenge)
  35. 35. Trajectory Planning Examples
  36. 36. How the Car Behaves
  37. 37. Overland Few road users, unobstructed view, dedicated lanes
  38. 38. Inner-City Denser traffic, obstructed view, dedicated lanes
  39. 39. Parked Cars Driving around static obstacles
  40. 40. Cyclists Driving around dynamic obstacles
  41. 41. Pedestrians Stopping for pedestrians at crosswalks
  42. 42. Intersections Stopping at traffic lights at intersections
  43. 43. Turning Observing cross and oncoming traffic (possible manual confirm)
  44. 44. Roundabouts Waiting for "empty slot" from a safe position
  45. 45. What Is Next
  46. 46. Complexity of Automation Low ego velocity High ego velocity Structured traffic environment Traffic Jam Highway Step1 Step 3 Chaotic traffic environment Parking Off-Highway Step 2 Step 4