An Event Data Recorder (or EDRs) serves as a blackbox or an accident data recorder for automobiles to record information related to vehicle crashes or accidents. Almost all vehicles in the US manufactured from 2006 have been fitted with event data recorders. EDRs work by continuously recording data such as vehicle acceleration and speed at the time of impact, braking, airbag deployment, seat-belt status, steering angle etc. The function of an EDR is to capture such events prior to an accident, at the time of the accident, and a few seconds after the accident. Usually audio and video data are excluded from EDRs. This serves as a history to determine the cause of an accident either due to fault in the vehicle or external circumstance or driver error. With the advent of compute engine SoCs that enable deep learning, object detection and tracking, and sensor fusion from sources such as camera, LiDAR, Radar, GPS, etc., it becomes imperative for an EDR to evolve in an autonomous self-driving system. The challenges faced range from not only capturing decisions, objects detected, position/navigation but also stitching a clear picture from sensors. All this has to be recorded synchronously in time along with system logs of various domain controllers such as Brake Control Unit, Steering Control unit, Vehicle Control Unit, Gateway, Digital Cockpit, and Autonomous control unit. The design challenges imposed by all this on an embedded real-time system will be explored in this talk. Standardization of tools to extract and interpret EDR data either from the cloud or the blackbox will be discussed as well.

1. Event Data Recorder (EDR)
for Self Driving Vehicles
Dr. Manish Kochhal, Ph.D.
Staff Software Engineer, Autonomous Drive Group
NIO Inc
opinions expressed are solely my own and do not express the views or opinions of my employer.

2. Outline of the
talk
What is an EDR?
Motivations
Implications of Self-Driving Cars on EDR
Capturing Data and
Decision Making
Sensor Fusion – Cameras, Radar, LiDAR,
Automatic Control - Drive-by-wire, Brake-
by-wire, Steering Assist
EDR Implementation – An Embedded and Distributed
Real-time System Design Challenge!
Standards for EDR format, tools etc.
opinions expressed are solely my own and do not express the views or opinions of my employer.

3. What is an EDR?
• EDR is a tiny blackbox embedded by automakers in the
airbag safety systems to record data around the time of
the accident.
• EDR data is usually kept secret and shared with NHTSA
engineers for accident reconstruction, if an
investigation occurs.
• EDR can also be analyzed for understanding crash
dynamics and improving vehicle safety.
• EDR blackboxes are not actually mandated.
• Privacy concerns have caused the NHTSA to advise
automakers only to capture tiny bits of information.
Image: autobody-review.com
opinions expressed are solely my own and do not express the views or opinions of my employer.

4. EDR – A causal record of events
around the time of the accident
Image: squarell.com
opinions expressed are solely my own and do not express the views or opinions of my employer.

5. EDR Data – In Non-AV Cars
Steering Wheel Angle - Steering wheel position, from 0º if wheel is in the neutral
position, to +/- 1080º if wheel is turned 3 times to either direction.
Lateral Acceleration - A measure of how hard the vehicle is cornering. Lateral
acceleration can also be impact induced.
Longitudinal Acceleration - A measure of how hard the vehicle is accelerating or
decelerating. Longitudinal acceleration can also be impact induced.
Yaw Rate - A measure of the rate at which the vehicle is changing its degree of
rotation. It is important to note that this measurement is a rate.
Source: Event Data Recorder (EDR) Reports: Understanding the Basics - Expert Article, Robson Forensic, The Expert.
opinions expressed are solely my own and do not express the views or opinions of my employer.

6. EDR Data - Reports
Roll Rate - A measure of the rate at which the vehicle is changing its degree of roll. It
is important to note that this measurement is a rate.
Delta-V - The change in velocity is calculated from accelerometer data in the Airbag Control Module and presented
for longitudinal and lateral directions. In order to determine actual Delta-V for the crash, engineers utilize both
datasets to determine the principle direction of force for the crash.
Source: Event Data Recorder (EDR) Reports: Understanding the Basics - Expert Article, Robson Forensic, The Expert.
opinions expressed are solely my own and do not express the views or opinions of my employer.

7. EDR Data – Countdown to crash
Source: Event Data Recorder (EDR) Reports: Understanding the Basics - Expert Article, Robson Forensic, The Expert.
opinions expressed are solely my own and do not express the views or opinions of my employer.

8. Sensors Publish event data to EDR over CAN
Source: Event Data Recorder (EDR) Position Statement, Association for the Advancement of Automotive Medicine
Design of ARM based Enhanced Event Data Recorder &Evidence Collecting System, N. P. Sirsikar, P. H. Chandankhede, IOSR-JECE, Volume 9, Issue 5, Ver. V (Sep - Oct. 2014)
opinions expressed are solely my own and do not express the views or opinions of my employer.

9. Levels of Vehicle Autonomy
Image: Automated Driving Systems, A Vision for Safety, NHTSA
opinions expressed are solely my own and do not express the views or opinions of my employer.

10. ASIL - Automotive Safety
Integrity Levels
• 5-level scale (QM, A, B, C, D)
• QM means, “standard Quality Assurance is
sufficient”
• From ASIL A onwards, additional risk reduction
actions must be taken
• ASIL D describes the highest risk potential.
• ASIL has requirements allocated to it.
• The defined safety goals are the top level safety
requirements (on vehicle level!)
opinions expressed are solely my own and do not express the views or opinions of my employer.
Source: https://semiwiki.com/eda/synopsys/7480-functional-safety-is-a-driving-topic-for-iso-26262/

11. NHTSA’s Guideline on Safety
for Autonomous Vehicles
• NHTSA has some requirements for AEB, LDW,
LKA etc. for L2+ autonomous cars.
• Following are still evolving:
• Federal and State Local Laws
• IEEE J3016 has defined what
Operational Design Domain (ODD)
means – however no accepted standard
as to what the ODDs are.
• Data Recording standards are defined
for L3, L4, and L5.
• EDR/Data Recording is important
• Assessing car/driver faults under
several ODD scenarios
• Fail to fallback or takeover by driver or
car or both.
opinions expressed are solely my own and do not express the views or opinions of my employer.

12. EDR of Sensing Data and
Decisions
Source: Delphi
opinions expressed are solely my own and do not express the views or opinions of my employer.

13. EDR needs to capture all
decisions
opinions expressed are solely my own and do not express the views or opinions of my employer.

14. A screenshot of what the Google car
sees approaching a right turn; inset,
the view from inside the car.
• The baseline image is the detailed area map in
grayscale.
• Layered atop that are objects identified by the
car's sensors, depicted in colorful geometric
boxes: purple for vehicles, red for cyclists,
yellow for pedestrians.
• The red and green ladders are objects that have
an immediate impact on the car's speed; in this
case, though the traffic light is green,
pedestrians prevent a turn, as does a cyclist
coming up on the right — in a spot a human
driver might easily miss.
• The flat green line shows the car's planned
route.
EDR needs to capture Path Planning,
perceived and predicted obstacles, lane
markers, construction zones, traffic signs
etc. in real-time.
opinions expressed are solely my own and do not express the views or opinions of my employer.

15. Predicting Bicyclist Intentions
• This is an example of a Cruise car trying to
predict the intentions of a bicyclist nearby.
• Such scenarios involve crunching a lot of
sensory inputs for a making real-time
conservative decision and accurate
predictions.
• The data, decisions, and predictions need to
be recorded in real-time in the EDR.
Complicated prediction of obstacles and
their movements need to be recorded.
opinions expressed are solely my own and do not express the views or opinions of my employer.

16. Advanced
BlackBox
Should be as advanced as avionics
flight data recorders.
Should be open, including the data
collected, so the truth of self driving
limitations is always available to make
the next car even safer.
Source: https://www.wardsauto.com/industry-voices/level-5-autonomy-it-ai-or-just-deep-programming
opinions expressed are solely my own and do not express the views or opinions of my employer.

17. Off-the-shelf Event Data
Recorders
• 5 Radars
• 4 Surround Cameras + 1 Trifocal Camera
• 1 LiDAR
• 4 Ultrasonic Sensor
• 1 IMU
• 1 GPS
• HD Maps
EDR: Capture data from 16+ sensors
opinions expressed are solely my own and do not express the views or opinions of my employer.

18. Design Components Involved
for EDR Development
• Mechanical design of EDR BlackBox
• E/E topology
• Distributed Communications Middleware –
clients publish interesting event data.
• Automotive Grade OS and Virtualization – to
make applications (including Data Recorder)
failsafe, redundant and reliable.
• Automotive Grade File System – data integrity
vital during read/writes.
• Wireless – upload to cloud
• Security – EDR cannot be hacked by
unauthorized users.
• Backup Power – available to save 10-15s of
post-crash EDR record
• Tools to extract EDR.
Automotive OS Features: Containment, Virtualization, Redundancy, Failover
opinions expressed are solely my own and do not express the views or opinions of my employer.

19. The Vehicle is a Distributed System
opinions expressed are solely my own and do not express the views or opinions of my employer.

20. EDR
Framework –
System
Requirements
•Urban city driving – unprotected left turns, pedestrian crossing
•Highway driving – lane detection, lane change,
•L2/L3+ features: AEB, ACC, LKAS, LDW, Assisted Parking, …
•AD Engagement, Disengagement, Decision Uncertainty, Driver
Attention, etc …
ODD Behavior Specification – specially for L4 and
L5 autonomy.
•A separate ASIL-D controller
•Supports high throughput network connectivity, 1Gbps.
•Traffic prioritization – Virtual LAN, flow prioritization based QoS
tags
•Safety Core in the EDR ECU – to support failover of recording
application
•High Throughput Read/Writes to in-memory records (Circular
Buffers)
•High Throughput Read/Writes to ASIL-D filesystem on primary and
backup storage arrays.
•PCIe or USB – automotive grade driver and stack to interface with
memory and storage.
•Storage and Memory Requirements to accommodate 16+ sensors
recording at varying rates for 10-15 seconds of pre, at, and post
accident events.
EDR ECU
opinions expressed are solely my own and do not express the views or opinions of my employer.

21. ODD Behavior – Classify Events severities: Normal,
Medium, High, Critical, Warning, Fault (its Context
Sensitive!)
ODD
Driver
behavior
Car behavior Environmental
behavior
Driver
engaged
Driver
Disengaged
Driver Assist
Driver
Manual
Driver Asleep AD Engaged
AD
Disengaged
Hands-OnHands-Off
Navigation
Change
Lane Status
Pedestrian
Objects
Detected
Potential
Obstacles
TSR - Status
Driver
Fallback
Object
Unclear
Safety
Decision
opinions expressed are solely my own and do not express the views or opinions of my employer.

22. Hypothetical ODD Event Scenario
• City Driving Scenario (Most Unpredictable!)
• AD Engaged –Normal Event
• Driver State, asleep - High
• Right turn from start to 2-way street - Normal Event
• Traffic Sign Recognition (TSR) event – STOP or Signal – Medium
• 4-way stop - High
• Decision to take an unprotected Left turn – Critical
• Unable to take a left turn – Fault
• Able to merge to a lane – Medium
• Automatic Emergency Braking – High
• Enter Construction Zone/Slow Speed – Medium
• Exit Construction Zone/Increase Speed – High
• Estimated Unprotected Pedestrian Crossing – Critical
• Enter Parking Lot – Medium
• Park – Normal
• Crash or Safety Stop or AD Disengaged or Driver Takeover – Critical
• Airbag Deployed – Fault
• Obstacle Classification – Estimate %age – Low, High, Medium, Critical
• Decision Certainty – Conservative, otherwise - Low, High, Medium, Critical
opinions expressed are solely my own and do not express the views or opinions of my employer.

23. EDR: Event Abstraction Framework
Event Clients Interface
(Register, Buffer, Trigger, Cancel, Trigger Status)
LiDAR Event
Client
Camera
Event Client
Navigation
Event Client
Radar Event
Client
Ultrasonic
Event Client
Driver State
Event Client
Decision Making
Event Client
Distributed Communications Middleware
Object Detection
Event Client
Event and its Attributes
(Priority, Name, Client Id, State)
Event
Context
Airbag Deployment
Event Client
AEB LDW ACC
opinions expressed are solely my own and do not express the views or opinions of my employer.

24. EDR: In-Memory Recording Framework, Circular
Buffering to our Rescue!
EDR
MCU
Decision
Camera
GPS/Maps
LiDAR
Radar
Gateway(CAN/ETH)
Pre
Post
Pre
Pre
At
PostPost
Pre
CB Organization
Primary CBs Secondary CBs
Tertiary CBs
opinions expressed are solely my own and do not express the views or opinions of my employer.

25. EDR – Blackbox storage on Distributed and
Parallel Filesystems
• TBs of data depending upon ODD
duration and complexity.
• We need highly reliable file-system.
This is a big data problem and there
are existing solutions in data center
space.
Source: Flash Memory in the emerging age of autonomy, S. Heinrich, Lucid Motors.
opinions expressed are solely my own and do not express the views or opinions of my employer.

26. Future Work: EDR Format,
Extraction and Visualization Tools
• Virtual Reality based Accident/Crash Reconstruction based on
BlackBox data pulled from the EDR.
• Pull the EDR from specific tools provided by the manufacturer.
• Convert it to a VR movie – again using the tools provided.
Image Source: Courtesy Google Images – Accident Reconstruction
opinions expressed are solely my own and do not express the views or opinions of my employer.

27. Existing
Questions in
Public Domain
In a fully autonomous car accident, the EDR data
may be the only operational driving record. Can
we trust the relationship between automakers
and investigators to allow the truth to be
revealed? How will we know what the car saw
before a crash with a human-driven car or
another bicyclist?
Currently, there is no independent infrastructure
established with guidelines in place to assess the
completeness of EDR data. However, there is
agreement for L2/L3 like driver assist features.
opinions expressed are solely my own and do not express the views or opinions of my employer.

28. Conclusions
General acceptable EDR content and format is a WIP
Need to get agreement from lawmakers, automotive
manufacturers, suppliers, for the EDR content
relevant to an ODD.
Customers need to be informed about the importance
of EDR.
EDR Events need to be specified for each of
autonomous driving ODD domains.
Similar to crash testing, guidelines should be set for
applicable ODDs and the EDR recorded. The data
recorded is validated.
opinions expressed are solely my own and do not express the views or opinions of my employer.