Your SlideShare is downloading. ×
Planning of a field operational test on navigation systems: Implementation and evaluation of pre-studies providing background information for the data analysis, Barbara Metz, Stefanie Schoch, Mariana Rakic & Friedemann Kuhn
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Planning of a field operational test on navigation systems: Implementation and evaluation of pre-studies providing background information for the data analysis, Barbara Metz, Stefanie Schoch, Mariana Rakic & Friedemann Kuhn


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Planning of a field operational test on navigation systems: Implementation and evaluation of pre-studies providing background information for the data analysis Barbara Metz, Stefanie Schoch, Mariana Rakic & Friedemann Kuhn
  • 2. Background
    • Presented research is part of the EC-funded project euroFOT:
      • In 2010 a field operational test will take place with the aim to evaluate the effect and usage of navigation systems
      • Driving without navigation system, with navigation systems provided by BMW & Daimler and with a mobile navigation system will be compared
    • General aim of euroFOT: Assess impact of investigated driver assistance systems on driving on a EU-level for a representative sample.
    • Two pre-studies will be presented which provide background information for planning and implementing the FOT-data analysis.
  • 3. Study 1: Questionnaire survey
    • Development of short questionnaire (5-10 minutes)
      • Usage and evaluation of navigation system
      • Frequency of driving & unfamiliar routes
      • Other driving relevant background information
    • Conduction Nov / Dec 2008 and June / July 2009
    • Sample:
      • N = 147 drivers at highway service area
      • N = 95 drivers at parking lot in front of supermarket
      • 30 % female, mean age 47 year (sd = 16.5)
  • 4. Study 1: Results
      • Of sample, 56% use / own navigation systems
      • Of users, 63% use a mobile device, 36% a built-in navigation system
      • There are relevant differences between owners and non-owners:
        • Owners of navigation systems have higher annual mileage
        • Owners of navigation systems drive more often on unfamiliar routes
        • Owners of navigation systems rate both system functions as more useful
  • 5. Study 1: Relevance for the FOT
    • Conduction of the FOT:
      • To enhance the likelihood of system effects, a sample should be recruited with high mileage drivers
        • Effect of navigation system probably mostly on unfamiliar trips
        • With higher mileage the likelihood of unfamiliar trips rises
        • Low mileage drivers only rarely experience the most relevant use case (unfamiliar route)
    • Analysis of the FOT-data (especially up-scaling of results for larger sample):
      • Results from FOT-sample can not be transferred linearly to larger sample
      • For up-scaling, the likelihood of unfamiliar trips has to be considered
  • 6. Study 2: Aims of the experiment
    • Part of pilot activities in euroFOT:
      • Test of data logging
      • Implementation of data base
      • Implementation of data analysis
        • Implementation and evaluation of algorithms for calculating derived measures (e.g. to detect certain events)
      • Evaluation of planned data analysis
        • Can the planned analysis differentiate between driving with and without navigation system?
  • 7. Study 2: Experimental design
    • Experiment with Daimler prototype vehicle for euroFOT
      • Data logging is comparable to FOT data
      • Logging of CAN-data, GPS-position and video data
    • Experimental design:
      • In real traffic
      • Defined circular route with N=10 different destinations
      • Drivers are not familiar with the destinations
      • N = 10 drivers, 5 with navigation system and 5 without navigation system
      • Experimenter protocols relevant events (e.g. turnarounds) during the drive
      • Duration 4-5 hours
  • 8. Study 2: Route
  • 9. Study 2: Implementing derived variables
    • For analysis of FOT-data, event detection algorithms are needed
      • Automated search for relevant events
      • If possible, no extensive control through video analysis necessary
    • Example: Turnaround manoeuvre
      • Manoeuvre, that directly relates to way finding performance
      • The experimenter protocolled turnarounds during the drive
      • In the CAN-data a turnaround is coded whenever reversing and steering wheel angle > 90° occur together within 60 seconds
  • 10. Study 2: Comparing protocols & event detection algorithm (Z=1.98, p<0.05) (Z=1.46 p=0.144)
  • 11. Study 2: Analysis of errors in event detection
    • Missed events (based on protocols):
      • Turning e.g. at intersections where no reversing is necessary
      • Can not be differentiated based on CAN-data from turning off at intersections
    • False positive events (based on analysis of video data):
      • 62 % of detected events are turnarounds
      • False positive are mostly parking manoeuvres (33%)
  • 12. Study 2: Improvement of algorithm
    • The evaluation of detected events helps to find several weakness of the used algorithm. For example:
      • Used algorithm was simple, but lead to events that endured about 60 seconds
        • Coded time window is too long (in 71% of events turning at next intersection is included)
  • 13. Study 2: Relevance for FOT
    • Experimental approach has several advantages for implementing the analysis for an FOT:
      • Because of the protocols, the algorithms for event detection can be evaluated regarding false positive and missed events
      • Weaknesses of the algorithms can be detected and improved
      • In case no improvement is possible, the impact of errors can be described for the data from the experiment
      • The planned analysis can be tested regarding its ability to detect effects of the tested system
      • The power of an purely automated analysis can be compared to an analysis relying on expert rating
  • 14. Conclusion
    • It has been tried to combine the expensive approach of an FOT with less costly methods:
    • A final evaluation of the contribution of the pre-studies is not possible before the analysis of the FOT-data.
      • Questionnaire survey provides information for a larger sample
      • Relevant for sample recruitment and planned up-scaling of results
      • Experiment provides well described and controlled data
      • Relevant for implementing and testing the analysis
  • 15. Thank you for your attention! For further information please contact [email_address]