1. The document discusses human-centered vehicle automation and the Advanced Vehicle Technology Consortium, which collects and analyzes data on advanced driver assistance systems and vehicle automation. 2. It provides examples of research on driver behavior with automated technologies, such as insights from Tesla Autopilot use and linking theories of driver-automation interaction to actual behavior. 3. The document advocates for developing new approaches for driver monitoring and attention management as vehicles incorporate more automation and changing how safety is evaluated to account for these technologies.

1. 1 © 2017 MIT AGELAB – PROPRIETARY AND CONFIDENTIAL.
© 2017 MIT AGELABCONTACT: REIMER@MIT.EDU
Bryan Reimer, Ph.D. | MIT AgeLab & New England University Transportation Center
Euro NCAP/IRCOBI Workshop on Safety of Automated Driving
September 12th, 2017
Human Centered Vehicle Automation
1

2. 2 © 2017 MIT AGELAB – PROPRIETARY AND CONFIDENTIAL.
Automatic transmissions reduce drivers’ operational workload.
Drivers have used the “freed up” resources to do other things!
We are talking about it more but the ship set sail decades ago!
Automation as an Evolving Process
Automatic
Increased DistractionManual

3. 3 © 2017 MIT AGELAB – PROPRIETARY AND CONFIDENTIAL.
• Originators: MIT AgeLab, Touchstone Evaluations & Agero
• Founding Members: Delphi, Liberty Mutual, Jaguar
Land Rover, Autoliv, Toyota
• Full Members: TBD
• Affiliate Members: Consumer Reports & TBD
• Focus: To collect and analyze cutting edge data that
objectively characterizes the behavioral and safety
benefit of advanced driver assistance systems, higher
levels of automation, and other in-vehicle technologies
under real-use conditions
The Advanced Vehicle Technology Consortium
Driver-Technology
Interaction Dynamic
An understanding of
system performance and
how drivers adapt to, use
(or do not use), and
behave with advanced
vehicle technologies
To develop
Looking Beyond the Technology

4. 4 © 2017 MIT AGELAB – PROPRIETARY AND CONFIDENTIAL.
Investigating Automated Technology Use in the Wild

5. 5 © 2017 MIT AGELAB – PROPRIETARY AND CONFIDENTIAL.
It is inconsistent, even for “trained drivers”
Terminology around Technology
Abraham, H., Seppelt, B., Mehler, B. & Reimer, B. (2017), What’s in a Name: Vehicle Technology Branding & Consumer Expectations for Automation. Paper
to appear in the Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicle Applications.

6. 6 © 2017 MIT AGELAB – PROPRIETARY AND CONFIDENTIAL.
Improved Volvo Pilot Assist performance with a dealer delivered upgrade.
Technologies are Improving Post-Production

7. 7 © 2017 MIT AGELAB – PROPRIETARY AND CONFIDENTIAL.
16,422 transfers of control across 454 hours of Autopilot use
• Human to machine
- From available – 7,308
- From ACC – 903
• Machine to human (human initiated)
- To manual – 6,911
- To ACC – 1,300
• Machine to human (system initiated)
- AI assistance required – 17
- ODD speed boundary (90 MPH) exceeded – 12
- Intersection (slow speed car following through turn) – 5
- Automated lane change (e.g. lane characteristic changes) – 4
- Other - 4
Insight on Tesla Autopilot Use
GPS points with Autopilot engaged (blue) overlaid on manual control

8. 8 © 2017 MIT AGELAB – PROPRIETARY AND CONFIDENTIAL.
When has a driver accepted a transition? regained full awareness?
Further, drivers demonstrate different use styles.
Linking Theory to Actual Behavior
≠
?
=
Seppelt, B.D. (2016). Conceptualizing the relationship between the driver and system for a transfer of control sequence.
TEI Technical Report No. 2016-1002. Detroit, MI: Touchstone Evaluations, Inc.

9. 9 © 2017 MIT AGELAB – PROPRIETARY AND CONFIDENTIAL.
Approach: End-to-end deep neural networks for perception and steering control
Using Naturalistic Data to Teach Robots to Drive
MIT 6.S094: Deep Learning for Self-Driving Cars - http://selfdrivingcars.mit.edu/

10. 10 © 2017 MIT AGELAB – PROPRIETARY AND CONFIDENTIAL.
Goal: Developing a deep reinforcement based a perceptual control system that
learns from experience how to avoid high speed crashes.
Teaching Cars to Learn

11. 11 © 2017 MIT AGELAB – PROPRIETARY AND CONFIDENTIAL.
Developing an Integrated Model of Driver Attention
Automation?
Managing Attention Through
State Support
More Information
• Broadening scientifically valid perspectives and
methodologies for the objective measurement of demand
placed on drivers by in-vehicle systems and technologies
during vehicle use, while considering the increasing role of
attention support and management.
• Moving the language of demand assessment from one
somewhat focused on distraction, to one that emphasizes
driver attention management and safe operation, such that
demands on driver, active safety systems, and other higher
order forms of automation can be considered as a whole.

12. 12 © 2017 MIT AGELAB – PROPRIETARY AND CONFIDENTIAL.
Developing New Approaches for Driver State Estimation
Application of computer vision and deep learning to several driver state detection tasks:
• Body position
• Gaze region estimation
• Emotion detection
• Cognitive load estimation
• Hand placement
• Smartphone detection and position estimation Image processing for cognitive load estimation
Fridman, L., Mehler, B., Reimer, B. & Freeman, W.T. (under review),
Cognitive load estimation in a large on-road driving dataset.

13. 13 © 2017 MIT AGELAB – PROPRIETARY AND CONFIDENTIAL.
Attention Management Perspective
AHEAD is doing research on the driver’s deployment of attention across time.
Attention Management Over Time
Figures adapted from © DENSO International America, Inc.
Distraction Perspective
An attention management perspective
allows us to ask the question:
“What were drivers doing upstream
that led to the downstream
consequence that is evident from the
distraction perspective” ? Why was
that last inopportune glance taken?
VS.

14. 14 © 2017 MIT AGELAB – PROPRIETARY AND CONFIDENTIAL.
Hybrid Measures of Attention Provide a More Comprehensive View of Safety
A larger loss of SA preceded
crash epochs in the 100 Car-Data
Seppelt, B.D., Seaman, S., Lee, J., Angell, L.S., Mehler, B., Reimer, B. (2017). Glass Half-Full: Predicting Crashes From Near-Crashes in The 100-Car Data using On-Road Glance Metrics.
Accident Analysis and Prevention.
Seaman, S., Lee, J., Seppelt, B., Angell, L., Mehler, B. & Reimer, B. (2017). It’s all in the timing: using the AttenD algorithm to assess texting in the NEST naturalistic driving database. Proceedings of the 9th
International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design.
Texting Crash epochs in the SHRP 2
NDS data start with depleted awareness
0.0
0.5
1.0
1.5
2.0
1 25 50 75 100
Percentage of Task Completed (%)
MeanAttenDBufferValue
Epoch Type
Texting - Baseline
Texting - Crash

15. 15 © 2017 MIT AGELAB – PROPRIETARY AND CONFIDENTIAL.
The Future May Be Autonomous, But…
Humans will continue to have a role for some time
• Level II systems introduce a number of use challenges
• Reported experiences with L1 and L2 technologies combined with
quantitative data tell a story
- Mode confusion
- Role confusion
- System confusion
• Investment in human centered engineering throughout the automotive
ecosystem is critical to a safe mobility future
- Improved communication with the driver
- Fused decision models that consider driver state
- Consumer education and training (dealer delivery and coaching)

16. 16 © 2017 MIT AGELAB – PROPRIETARY AND CONFIDENTIAL.
NCAP May Need to Adapt to:
• Technology enhancements post production
• Active learning systems
• Automation that changes how we view harm (fatalities, injuries,
property damage, safety, environmental impacts, etc.)
• Considerations of safety from various perspectives in benefit estimation
- Societal
- Engineering (e.g. suppliers and OEMs)
- Users
• New cars will maintain their form factor for some time, but changes will
appear that could justify the elimination of:
- The steering wheel, throttle, and brake
- Some of “today’s” passive safety systems
- Traditional manual controlled vehicles in some contexts

17. 17 © 2017 MIT AGELAB – PROPRIETARY AND CONFIDENTIAL.
Vehicle Miles Traveled (VMT) Vehicle Miles Driven (VMD)
The Future May Be One of Relatively
“Novice” Drivers
Today
VMT ≈ VMD
Tomorrow?
VMT ≠ VMD