Talk made during Transportation Research Arena 2014 (Paris). This talk present the lessons learned by the French team during the first Large Scale Field Operational Test in Europe (EuroFOT).
Market Analysis in the 5 Largest Economic Countries in Southeast Asia.pdf
Speed regulation systems evaluation the eurofot example
1. Impact evaluation of speed
regulation systems using naturalistic
driving data: The EuroFOT example.
Saint Pierre, Guillaume, IFSTTAR, France
Tattegrain, Hélène, IFSTTAR, France
Val, Clément, CEESAR, France
* guillaume.saintpierre@ifsttar.fr
STS N°48 TRA2014 Paris 14-17 avril 2014
2. Saint Pierre G., Tattegrain H., Val C.
Introduction
Increasing penetration of driving assistance systems
Needs to measure theirs impacts during real uses
Several projects launched recently (FP7 funded)
Field operational tests (FOT)
FESTA methodology
Naturalistic driving data
Many challenges were adressed, and many lessons
learned
Let’s come back to the french EuroFOT experience
The first large scale FOT, ended in 2012
STS N°48 TRA2014 Paris 14-17 avril 2014 2
4. Saint Pierre G., Tattegrain H., Val C.
EuroFOT in France
VMC handled by
CEESAR
• 35 drivers using their
own car in the west of
Paris, during 6 months
• Light instrumentation
5 identical cars
replaced the subject
ones three times
Full instrumentation
(incl. Video)
545 000 km of data
analysed
STS N°48 TRA2014 Paris 14-17 avril 2014 4
Low Level High Level
Vehicles used 35 drivers’
owned
vehicles.
5 vehicles owned by
CEESAR and loaned to
participants
CTAG datalogger 2
Max 4 CAN Channels
GPS
GPRS data transfer
●
2 channels
used
●
●
●
4 channels used
●
(not used : manual
transfer)
TRW AC20 radar
(not part of standard vehicle
equipment)
● ●
VideoLogger
(custom made for CEESAR,
H.264)
●
Cameras
(B&W, SuperHAD Exview)
4
Mobileye AWS
(added, with special firmware)
●
Smarteye Eyetracker ●
5. Saint Pierre G., Tattegrain H., Val C.
Experimental design
Dream
Reality
STS N°48 TRA2014 Paris 14-17 avril 2014 5
Questionnaires were
administred 4 times
Rotation of fully
equiped vehicules
among participants
Month
1
Month
2
Month
3
Month
4
Month
5
Month
6
Month
7
Month
8
Month
9
Month
10
Month
11
Month
12
Baseline Treatment Treatment
Screening,
Time 1
Time 2 Time 3 (a) Time 3 (b)
Time 4,
Debriefing
6. Saint Pierre G., Tattegrain H., Val C.
Lessons learned (1)
Recruitment needs car owners database
acces to be efficient
GPRS data transfert problematic, consider
UMTS instead,
Simplify experimental plan
NDS style
« instrument and forget »
STS N°48 TRA2014 Paris 14-17 avril 2014 6
7. Saint Pierre G., Tattegrain H., Val C.
Some issues for ND data
Data reduction
• Reduce or aggregate continuous data to a significant level
Data modelling
• Avoid comparisons between heterogeneous datasets
• Control for exposure
• Take into account the intrinsic correlation present in the
data (repeated measures framework)
Deal with rare events
• Post processing detection of “safety related events”
Results extrapolation
• Transform events based analyses into casualties reduction
STS N°48 TRA2014 Paris 14-17 avril 2014 7
8. Saint Pierre G., Tattegrain H., Val C.
EuroFOT « solutions »
Data reduction
Identify homogeneous sections of data
Split sections in identical time epochs (10-30 sec.)
Data modelling
Use suitable statistical models (GEE, GLMM, instead of ANOVA)
Produce Odds ratios results
Deal with rare events
Automatically detect candidates events (triggers, system use
etc...)
Confirm identification by video + Annotation
Extract corresponding baseline and do some stats...
Results extrapolation
Speed & Accidents relationships
STS N°48 TRA2014 Paris 14-17 avril 2014 8
9. Saint Pierre G., Tattegrain H., Val C.
Key results
Behavior, acceptance, usage
CC usage does not vary significantly over time.
SL usage does not vary significantly over time.
Drivers tend to use more one of the two systems.
CC usage favorable driving conditions
SL usage adverse conditions (ex. Night)
STS N° TRA2014 Paris 14-17 avril 2014 9
10. Saint Pierre G., Tattegrain H., Val C.
Safety: Events based
analysis
STS N° TRA2014 Paris 14-17 avril 2014 10
Safety events are rare: Odds ratios can be interpreted as
relative risk
SL associated with less frequent safety related events (SRE)
CC associated with less SRE, except over-speeding
11. Saint Pierre G., Tattegrain H., Val C.
Lessons learned (2)
Baseline selection/definition for each RQ
hypothesis is crucial
Needs to control for external factors (traffic, visibility)
A data aggregation method is needed
It has an impact on the analysis
Scaling up proove to be very difficult
Various methods tried during FOTs
None is perfect
Events based analysis (EBA)
applicable to any system which impact is related to
the occurrence of this event
STS N°48 TRA2014 Paris 14-17 avril 2014 11
12. Saint Pierre G., Tattegrain H., Val C.
Conclusion & recommendations
for future FOT
Why using NDS ?
To get precise estimates of safety related events frequency
(with/without system)
Identify systems usage context
Identify systems misuses and potential countermeasures
Limitations
Difficult to get a representative panel
Very hard to extrapolate to casualties reductions
Further works
Identify important measures for road safety
Increase panel size and representativity
Define/quantify safety critical events
STS N°48 TRA2014 Paris 14-17 avril 2014 12
13. Saint Pierre G., Tattegrain H., Val C.STS N°48 TRA2014 Paris 14-17 avril 2014 13
Thank you for your attention
Guillaume SAINT PIERRE
Guillaume.saintpierre@ifsttar.fr
COSYS/LIVIC
Components & systems department
Interaction vehicles/drivers/infrastructure research unit