Downloaded 13 times
More Related Content
Similar to Event Data Recorder (EDR) for Self Driving Vehicles
Event Data Recorder (EDR) for Self Driving Vehicles
- 1. Event Data Recorder(EDR) for Self Driving Vehicles Dr. Manish Kochhal, Ph.D. Staff Software Engineer, Autonomous Drive Group NIO Inc opinions expressed are solely my own and do not express the views or opinions of my employer.
- 2. Outline of the talk Whatis an EDR? Motivations Implications of Self-Driving Cars on EDR Capturing Data and Decision Making Sensor Fusion – Cameras, Radar, LiDAR, Automatic Control - Drive-by-wire, Brake- by-wire, Steering Assist EDR Implementation – An Embedded and Distributed Real-time System Design Challenge! Standards for EDR format, tools etc. opinions expressed are solely my own and do not express the views or opinions of my employer.
- 3. What is anEDR? • EDR is a tiny blackbox embedded by automakers in the airbag safety systems to record data around the time of the accident. • EDR data is usually kept secret and shared with NHTSA engineers for accident reconstruction, if an investigation occurs. • EDR can also be analyzed for understanding crash dynamics and improving vehicle safety. • EDR blackboxes are not actually mandated. • Privacy concerns have caused the NHTSA to advise automakers only to capture tiny bits of information. Image: autobody-review.com opinions expressed are solely my own and do not express the views or opinions of my employer.
- 4. EDR – Acausal record of events around the time of the accident Image: squarell.com opinions expressed are solely my own and do not express the views or opinions of my employer.
- 5. EDR Data –In Non-AV Cars Steering Wheel Angle - Steering wheel position, from 0º if wheel is in the neutral position, to +/- 1080º if wheel is turned 3 times to either direction. Lateral Acceleration - A measure of how hard the vehicle is cornering. Lateral acceleration can also be impact induced. Longitudinal Acceleration - A measure of how hard the vehicle is accelerating or decelerating. Longitudinal acceleration can also be impact induced. Yaw Rate - A measure of the rate at which the vehicle is changing its degree of rotation. It is important to note that this measurement is a rate. Source: Event Data Recorder (EDR) Reports: Understanding the Basics - Expert Article, Robson Forensic, The Expert. opinions expressed are solely my own and do not express the views or opinions of my employer.
- 6. EDR Data -Reports Roll Rate - A measure of the rate at which the vehicle is changing its degree of roll. It is important to note that this measurement is a rate. Delta-V - The change in velocity is calculated from accelerometer data in the Airbag Control Module and presented for longitudinal and lateral directions. In order to determine actual Delta-V for the crash, engineers utilize both datasets to determine the principle direction of force for the crash. Source: Event Data Recorder (EDR) Reports: Understanding the Basics - Expert Article, Robson Forensic, The Expert. opinions expressed are solely my own and do not express the views or opinions of my employer.
- 7. EDR Data –Countdown to crash Source: Event Data Recorder (EDR) Reports: Understanding the Basics - Expert Article, Robson Forensic, The Expert. opinions expressed are solely my own and do not express the views or opinions of my employer.
- 8. Sensors Publish eventdata to EDR over CAN Source: Event Data Recorder (EDR) Position Statement, Association for the Advancement of Automotive Medicine Design of ARM based Enhanced Event Data Recorder &Evidence Collecting System, N. P. Sirsikar, P. H. Chandankhede, IOSR-JECE, Volume 9, Issue 5, Ver. V (Sep - Oct. 2014) opinions expressed are solely my own and do not express the views or opinions of my employer.
- 9. Levels of VehicleAutonomy Image: Automated Driving Systems, A Vision for Safety, NHTSA opinions expressed are solely my own and do not express the views or opinions of my employer.
- 10. ASIL - AutomotiveSafety Integrity Levels • 5-level scale (QM, A, B, C, D) • QM means, “standard Quality Assurance is sufficient” • From ASIL A onwards, additional risk reduction actions must be taken • ASIL D describes the highest risk potential. • ASIL has requirements allocated to it. • The defined safety goals are the top level safety requirements (on vehicle level!) opinions expressed are solely my own and do not express the views or opinions of my employer. Source: https://semiwiki.com/eda/synopsys/7480-functional-safety-is-a-driving-topic-for-iso-26262/
- 11. NHTSA’s Guideline onSafety for Autonomous Vehicles • NHTSA has some requirements for AEB, LDW, LKA etc. for L2+ autonomous cars. • Following are still evolving: • Federal and State Local Laws • IEEE J3016 has defined what Operational Design Domain (ODD) means – however no accepted standard as to what the ODDs are. • Data Recording standards are defined for L3, L4, and L5. • EDR/Data Recording is important • Assessing car/driver faults under several ODD scenarios • Fail to fallback or takeover by driver or car or both. opinions expressed are solely my own and do not express the views or opinions of my employer.
- 12. EDR of SensingData and Decisions Source: Delphi opinions expressed are solely my own and do not express the views or opinions of my employer.
- 13. EDR needs tocapture all decisions opinions expressed are solely my own and do not express the views or opinions of my employer.
- 14. A screenshot ofwhat the Google car sees approaching a right turn; inset, the view from inside the car. • The baseline image is the detailed area map in grayscale. • Layered atop that are objects identified by the car's sensors, depicted in colorful geometric boxes: purple for vehicles, red for cyclists, yellow for pedestrians. • The red and green ladders are objects that have an immediate impact on the car's speed; in this case, though the traffic light is green, pedestrians prevent a turn, as does a cyclist coming up on the right — in a spot a human driver might easily miss. • The flat green line shows the car's planned route. EDR needs to capture Path Planning, perceived and predicted obstacles, lane markers, construction zones, traffic signs etc. in real-time. opinions expressed are solely my own and do not express the views or opinions of my employer.
- 15. Predicting Bicyclist Intentions •This is an example of a Cruise car trying to predict the intentions of a bicyclist nearby. • Such scenarios involve crunching a lot of sensory inputs for a making real-time conservative decision and accurate predictions. • The data, decisions, and predictions need to be recorded in real-time in the EDR. Complicated prediction of obstacles and their movements need to be recorded. opinions expressed are solely my own and do not express the views or opinions of my employer.
- 16. Advanced BlackBox Should be asadvanced as avionics flight data recorders. Should be open, including the data collected, so the truth of self driving limitations is always available to make the next car even safer. Source: https://www.wardsauto.com/industry-voices/level-5-autonomy-it-ai-or-just-deep-programming opinions expressed are solely my own and do not express the views or opinions of my employer.
- 17. Off-the-shelf Event Data Recorders •5 Radars • 4 Surround Cameras + 1 Trifocal Camera • 1 LiDAR • 4 Ultrasonic Sensor • 1 IMU • 1 GPS • HD Maps EDR: Capture data from 16+ sensors opinions expressed are solely my own and do not express the views or opinions of my employer.
- 18. Design Components Involved forEDR Development • Mechanical design of EDR BlackBox • E/E topology • Distributed Communications Middleware – clients publish interesting event data. • Automotive Grade OS and Virtualization – to make applications (including Data Recorder) failsafe, redundant and reliable. • Automotive Grade File System – data integrity vital during read/writes. • Wireless – upload to cloud • Security – EDR cannot be hacked by unauthorized users. • Backup Power – available to save 10-15s of post-crash EDR record • Tools to extract EDR. Automotive OS Features: Containment, Virtualization, Redundancy, Failover opinions expressed are solely my own and do not express the views or opinions of my employer.
- 19. The Vehicle isa Distributed System opinions expressed are solely my own and do not express the views or opinions of my employer.
- 20. EDR Framework – System Requirements •Urban citydriving – unprotected left turns, pedestrian crossing •Highway driving – lane detection, lane change, •L2/L3+ features: AEB, ACC, LKAS, LDW, Assisted Parking, … •AD Engagement, Disengagement, Decision Uncertainty, Driver Attention, etc … ODD Behavior Specification – specially for L4 and L5 autonomy. •A separate ASIL-D controller •Supports high throughput network connectivity, 1Gbps. •Traffic prioritization – Virtual LAN, flow prioritization based QoS tags •Safety Core in the EDR ECU – to support failover of recording application •High Throughput Read/Writes to in-memory records (Circular Buffers) •High Throughput Read/Writes to ASIL-D filesystem on primary and backup storage arrays. •PCIe or USB – automotive grade driver and stack to interface with memory and storage. •Storage and Memory Requirements to accommodate 16+ sensors recording at varying rates for 10-15 seconds of pre, at, and post accident events. EDR ECU opinions expressed are solely my own and do not express the views or opinions of my employer.
- 21. ODD Behavior –Classify Events severities: Normal, Medium, High, Critical, Warning, Fault (its Context Sensitive!) ODD Driver behavior Car behavior Environmental behavior Driver engaged Driver Disengaged Driver Assist Driver Manual Driver Asleep AD Engaged AD Disengaged Hands-OnHands-Off Navigation Change Lane Status Pedestrian Objects Detected Potential Obstacles TSR - Status Driver Fallback Object Unclear Safety Decision opinions expressed are solely my own and do not express the views or opinions of my employer.
- 22. Hypothetical ODD EventScenario • City Driving Scenario (Most Unpredictable!) • AD Engaged –Normal Event • Driver State, asleep - High • Right turn from start to 2-way street - Normal Event • Traffic Sign Recognition (TSR) event – STOP or Signal – Medium • 4-way stop - High • Decision to take an unprotected Left turn – Critical • Unable to take a left turn – Fault • Able to merge to a lane – Medium • Automatic Emergency Braking – High • Enter Construction Zone/Slow Speed – Medium • Exit Construction Zone/Increase Speed – High • Estimated Unprotected Pedestrian Crossing – Critical • Enter Parking Lot – Medium • Park – Normal • Crash or Safety Stop or AD Disengaged or Driver Takeover – Critical • Airbag Deployed – Fault • Obstacle Classification – Estimate %age – Low, High, Medium, Critical • Decision Certainty – Conservative, otherwise - Low, High, Medium, Critical opinions expressed are solely my own and do not express the views or opinions of my employer.
- 23. EDR: Event AbstractionFramework Event Clients Interface (Register, Buffer, Trigger, Cancel, Trigger Status) LiDAR Event Client Camera Event Client Navigation Event Client Radar Event Client Ultrasonic Event Client Driver State Event Client Decision Making Event Client Distributed Communications Middleware Object Detection Event Client Event and its Attributes (Priority, Name, Client Id, State) Event Context Airbag Deployment Event Client AEB LDW ACC opinions expressed are solely my own and do not express the views or opinions of my employer.
- 24. EDR: In-Memory RecordingFramework, Circular Buffering to our Rescue! EDR MCU Decision Camera GPS/Maps LiDAR Radar Gateway(CAN/ETH) Pre Post Pre Pre At PostPost Pre CB Organization Primary CBs Secondary CBs Tertiary CBs opinions expressed are solely my own and do not express the views or opinions of my employer.
- 25. EDR – Blackboxstorage on Distributed and Parallel Filesystems • TBs of data depending upon ODD duration and complexity. • We need highly reliable file-system. This is a big data problem and there are existing solutions in data center space. Source: Flash Memory in the emerging age of autonomy, S. Heinrich, Lucid Motors. opinions expressed are solely my own and do not express the views or opinions of my employer.
- 26. Future Work: EDRFormat, Extraction and Visualization Tools • Virtual Reality based Accident/Crash Reconstruction based on BlackBox data pulled from the EDR. • Pull the EDR from specific tools provided by the manufacturer. • Convert it to a VR movie – again using the tools provided. Image Source: Courtesy Google Images – Accident Reconstruction opinions expressed are solely my own and do not express the views or opinions of my employer.
- 27. Existing Questions in Public Domain Ina fully autonomous car accident, the EDR data may be the only operational driving record. Can we trust the relationship between automakers and investigators to allow the truth to be revealed? How will we know what the car saw before a crash with a human-driven car or another bicyclist? Currently, there is no independent infrastructure established with guidelines in place to assess the completeness of EDR data. However, there is agreement for L2/L3 like driver assist features. opinions expressed are solely my own and do not express the views or opinions of my employer.
- 28. Conclusions General acceptable EDRcontent and format is a WIP Need to get agreement from lawmakers, automotive manufacturers, suppliers, for the EDR content relevant to an ODD. Customers need to be informed about the importance of EDR. EDR Events need to be specified for each of autonomous driving ODD domains. Similar to crash testing, guidelines should be set for applicable ODDs and the EDR recorded. The data recorded is validated. opinions expressed are solely my own and do not express the views or opinions of my employer.