This document discusses tools and processes for analyzing field operational test (FOT) data. It provides an overview of the SAFER analysis platform, which processes complex driving data from multiple sources through several steps: decryption, synchronization, resampling, quality calculation, and event generation. The data is then analyzed to study driver behavior, safety systems, and more. Further development needs include reducing processing time, cloud storage, efficient data structures, automatic video coding, and anonymization solutions while retaining valuable information.

1. HOW YOU USE THE
DATA FOR FOT
ANALYSIS
Helena Gellerman, SAFER

2. Content
 SAFER overview
 Present tools and processes in the FOT
analysis platform
 Further development needs for FOT/Pilot data
analysis

3. Swedish Transport Administration
Swedish Transport Agency
Region Västra Götaland
City of Gothenburg
AB Volvo
Autoliv
ÅF
Folksam
If
Lindholmen Science Park
Scandinavian Automotive Suppliers
Scania
Sweco
Volvo Car Corporation
IRezQ
Malmeken
Chalmers University of Technology
University of Gothenburg
Halmstad University
KTH
Lund University
Acreo
SP
Swerea IVF
Swerea SICOMP
TÖI
Viktoria institute
VTI
Borås University
Skövde University
SAFERVehicle and Traffic Safety Centre
30 partners in collaboration
Society
IndustryAcademy &
Institutes

4. SAFER research programmes
Crash
Traffic Safety Analysis
Impact
Post-crashPre-crash
Each programme governed by reference group

5. SAFER FOT/NDS activities 2006-2015

6. Present tools and processes
in the FOT analysis platform

7. Driver and forward camera
Rear cameraEye/head tracker
Logger
Feet camera
Field Operational Test
Equipment

8. Security and analysis platform
Video
Data
Data owners (e.g.
SAFER + OEM)
Request
OK
D e c r y p t d a t a ,
e x t ra c t a ll d a t a
s o u rc e s a n d
s a v e i n
i n t e rm e d ia t e
R a w M a t . m a t
f o r m a t
P e rf o r m p r o c e s s i n g o f
s o m e O E M s e n s it i v e
m e a s u re s t o l e s s s e n s i t iv e
d e ri v e d m e a s u r e s ( p re -p r e -
p r o c e s s i n g o n C A N ) .
I n c l u d e e x t r a c t o f
b a s e l in e / t r e a t e m e n t .
D i s k p lu g g e d i n t o
U S B o n
w o rk s t a t i o n .
M a t la b s c ri p t i n i t i a t e d .
P e r t rip p r o c e s s i n g
s t a rt s . E s t im a t e s p a c e
n e e d e d a n d c h e c k f re e
s p a c e o n t r a n s f e r d i s k s
L o a d v e h ic le
u n iq u e
c o n f ig u r a t io n f i le
a n d c re a t e o u t p u t
d i re c t o r i e s i f
n e c e s s a ry .
T o o s h o r t
t ri p s a r e
d i s c a rd e d
( < X s )
D e c o d e
C A N i n t o
m e a s u r e s
(h u m a n
re a d a b le )
F i x 1 )
t im e s t a m p s
p e r d a t a s o u rc e .
U s e l in e a r
re g re s s i o n t o
a v o i d c l o c k d ri f t
C a l c u la t e
1 )
p r e -
re s a m p le d e r iv e d
m e a s u re s (e . g J e r k ) p e r
d a t a s o u rc e . A d d t o
m e a s u r e s f o r t h is d a t a
s o u r c e i n o D a t a S e t .
I n s t a n t ia t e 1 )
o D B d a t a (r e s u lt
M a t la b f o rm a t )
f o r t h e f i rs t
t i m e .
A d d
m e t a d a t a
1 )
i n f o rm a t io n
t o t h e
o D B d a t a
C r e a t e a c o m m o n
t i m e v e c t o r a t
1 0 H z b a s e d o n
t h e C A N -v e lo c it y
t i m e s t a m p s
R e m o v e t h e f i rs t
5 s 1 )
a n d t h e la s t
5 s 1 )
o f d a t a d u e t o
s t a r tu p / s h u t d o w n
e f f e c t s o n d a t a
s o u rc e s
S a v e t h e f i le i n
t h e in t e r m e d ia t e
fo r m a t
R a w M a t . m a t
C h a n g e m e a s u re
n a m e s a c c o r d i n g
t o c o n f i g u r a t io n
f il e
1 )
(n a m e
h a r m o n iz a ti o n )
A p p ly
1 )
a n t i -
a li a s i n g f il t e r.
A d d m e t a d a t a 1 )
a b o u t f il t e r a n d
re s a m p le d d a t a t o
o D B d a t a
A p p l y re s a m p li n g 1 )
f o r e a c h in d i v id u a l
m e a s u r e s . A d d t o o D B d a t a . m a t (p r e -
p ro c e s s in g re s u l t f o r m a t )
A d d c o m m o n t im e
in f o r m a ti o n f ro m
r e s a m p l in g t o o D B d a t a
(o n e t i m e o n ly )
R u n 1 )
a f i rs t d a ta
v e ri f ic a t io n s c r ip t o n
o D B d a t a . .
C a lc u la t e
1 )
p e r-s a m p le
q u a li t y o n o D B d a t a f o r a
f e w m e a s u r e s / d a t a
s o u rc e s .
C a l c u l a t e
1 )
p e r- m e a s u r e
q u a l i t y o n o D B d a t a f o r
m e a s u re s , u s i n g p e r -
s a m p l e q u a l it y .
C a lc u l a t e 1 )
p e r- t rip
q u a l it y . B a s e d o n p e r
m e a s u re q u a l it y .
C a lc u la t e 1 )
a ll
d e ri v e d
m e a s u r e s .
C a lc u la t e i n t ie rs .
C a lc u la t e
1 )
a l l
e v e n t s (t i e r
b a s e d ) .
C a l c u l a t e 1 )
d e r iv e d
m e a s u re
q u a li t y
C a lc u la t e 1 )
K a l m a n
f i lt e r s e n s o r f u s i o n
f o r p o s i t i o n a n d
h e a d i n g
e n h a n c e m e n t .
H a n d le 1 )
s ig n i f ic a n t d ig i t s
a n d t o o l a rg e /
in f i n it e v a lu e s .
A d d d ri v e r I D
S o rt m e a s u r e s
a n d e v e n t s
a lp h a b e t i c a l ly
( s i m p li f y u s a g e )
S a v e
o D B d a t a o n
s e rv e r a n d
t ra n s fe r d is k .
C h e c k t h a t a l l f i l e s
h a v e b e e n
p ro c e s s e d a n d
r e m o v e d a t a f ro m
o rig in a l d i s k
W h e n t ra n f e r d i s k
is a lm o s t f u ll ,
m o v e t o S A F E R /
C h a lm e r s f o r
u p lo a d i n g
P l u g t ra n s fe r d is k
i n t o U p lo a d i n g
s t a t i o n a t S A F E R
a n d s t a rt D a t a
u p l o a d
C h e c k t h a t a ll f i le s
h a v e b e e n
u p l o a d e d a n d
re m o v e d a ta f r o m
t ra n s f e r d i s k
M o v e t r a n s f e r
d i s k b a c k t o d a ta
p ro v id e rP h y s ic a l m o v e
o f d i s k
1 )
B a s e d o n t h e p e r -O E M i n f o rm a t io n i n t h e M S E x c e l d o c u m e n t c a l le d M E P S . T h i s f i le i s p a rs e d o n M a t l a b a n d a . m a t -f i le c a ll e d o P r e P r o c C f g . m a t
( c o n f ig u r a t io n f i le p e r v e h i c l e ) is u s e d t h o u g h o u t t h e p re - p ro c e s s i n g .
6.Savingand
uploadingof
data
5.Calculate
derived
measures
andevents
4.Calculate
quality
3.Filterand
resampleper
measure
2.Save,fixtime
andhighfreq
derivedmeasures
1.Readand
decypt/decode
data
D is k p l u g g e d i n t o U S B
o n w o r k s t a t io n a n d
f i le s a re c o p i e d
( s e p a ra t p r o c e s s f r o m
t h e o th e r p re -
p ro c e s s i n g . )
I d e n t i f i c a t io n
i f t ri p is
b a s e li n e o r
t re a t m e n t
D o M A P -
m a t c h i n g
(d a t a
e n r ic h m e n t )
a n d a d d t o
R a w M a t

9. Data
 Complex data generated and collected from
o In-vehicle network (CAN, LIN, MOST)
o Sensors (accelerometers, head tracker, eye tracker)
o GPS
o Cameras
o Communication data (V2X)
 Subjective data (questionairs, manual video
annotations
 Contextual data (weather, mapdata)
 75000 hours of driving data plus estimated
200000 hours from EU project UDRIVE

10. Data processing steps using
HPC
 Decryption
 Synchronization
 Re-sampling
 Harmonization
 Creating derived measures
 Pre-computed event generation
 Data hosted in databases (Oracle/MySQL)
and files for videos

11. Data analysis
 Depending on the analysis
 Frequently used steps are
o Using pre-computed events or
o Creating new definitions of
events from all datasets
o Sandboxing on a subset of the
database
o Validating data quality manually
o Running algorithm on full
dataset
o Manual coding of events
o Final analysis

12. Research areas
 Driver behaviour
 Crash causation
 Impact of new active safety systems
 Intersection safety
 Infrastructure design
 Mobility
 Eco-driving
 Development of driver models  Automation

13. Further development needs for
FOT/Pilot data analysis
Reduce the time for data transfer/mgmt/processing –
research  real-time
Data sent over gprs/wifi to cloud based storage
Principles for edge computing / data abstraction /
aggregation
Efficient data structures – efficient data extraction
Visualisation tools – data mining and analysis results
Automatic video coding – today manual annotations
Open(?) repositories with high quality context information
(maps, weather, traffic conditions)

14. Further development needs
for FOT/Pilot data analysis
Personal integrity
Anonymization without loosing valuable information
Research  real time anonymization
Automated vehicles operations and data collection

Solutions:
Data anonymization
Automatic video coding
Open, public data  data in secure enclaves

15. Thank you for your attention
Contact info:
Helena Gellerman
Area manager FOT/NDS at SAFER
FOTNet Data –
Data Sharing Framework WP leader
helena.gellerman@chalmers.se
+46 31 7721095
+46 761 191429