Planning of a field operational test on navigation systems:   Implementation and evaluation of pre-studies providing backg...
Background <ul><li>Presented research is part of the EC-funded project euroFOT: </li></ul><ul><ul><li>In 2010 a field oper...
Study 1: Questionnaire survey <ul><li>Development of short questionnaire (5-10 minutes) </li></ul><ul><ul><li>Usage and ev...
Study 1: Results <ul><ul><li>Of sample, 56% use / own navigation systems </li></ul></ul><ul><ul><li>Of users, 63% use a mo...
Study 1: Relevance for the FOT <ul><li>Conduction of the FOT: </li></ul><ul><ul><li>To enhance the likelihood of system ef...
Study 2: Aims of the experiment <ul><li>Part of pilot activities in euroFOT: </li></ul><ul><ul><li>Test of data logging </...
Study 2: Experimental design <ul><li>Experiment with Daimler prototype vehicle for euroFOT </li></ul><ul><ul><li>Data logg...
Study 2: Route
Study 2: Implementing derived variables <ul><li>For analysis of FOT-data, event detection algorithms are needed </li></ul>...
Study 2: Comparing protocols & event detection algorithm (Z=1.98, p<0.05)   (Z=1.46 p=0.144)
Study 2: Analysis of  errors in event detection <ul><li>Missed events (based on protocols): </li></ul><ul><ul><li>Turning ...
Study 2: Improvement of algorithm <ul><li>The evaluation of detected events helps to find several weakness of the used alg...
Study 2: Relevance for FOT <ul><li>Experimental approach has several advantages for implementing the analysis for an FOT: ...
Conclusion <ul><li>It has been tried to combine the expensive approach of an FOT with less costly methods: </li></ul><ul><...
Thank you for your attention! For further information please contact [email_address] www.izvw.de
Upcoming SlideShare
Loading in …5
×

Planning of a field operational test on navigation systems: Implementation and evaluation of pre-studies providing background information for the data analysis, Barbara Metz, Stefanie Schoch, Mariana Rakic & Friedemann Kuhn

1,042 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,042
On SlideShare
0
From Embeds
0
Number of Embeds
84
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Planning of a field operational test on navigation systems: Implementation and evaluation of pre-studies providing background information for the data analysis, Barbara Metz, Stefanie Schoch, Mariana Rakic & Friedemann Kuhn

  1. 1. Planning of a field operational test on navigation systems: Implementation and evaluation of pre-studies providing background information for the data analysis Barbara Metz, Stefanie Schoch, Mariana Rakic & Friedemann Kuhn
  2. 2. Background <ul><li>Presented research is part of the EC-funded project euroFOT: </li></ul><ul><ul><li>In 2010 a field operational test will take place with the aim to evaluate the effect and usage of navigation systems </li></ul></ul><ul><ul><li>Driving without navigation system, with navigation systems provided by BMW & Daimler and with a mobile navigation system will be compared </li></ul></ul><ul><li>General aim of euroFOT: Assess impact of investigated driver assistance systems on driving on a EU-level for a representative sample. </li></ul><ul><li>Two pre-studies will be presented which provide background information for planning and implementing the FOT-data analysis. </li></ul>
  3. 3. Study 1: Questionnaire survey <ul><li>Development of short questionnaire (5-10 minutes) </li></ul><ul><ul><li>Usage and evaluation of navigation system </li></ul></ul><ul><ul><li>Frequency of driving & unfamiliar routes </li></ul></ul><ul><ul><li>Other driving relevant background information </li></ul></ul><ul><li>Conduction Nov / Dec 2008 and June / July 2009 </li></ul><ul><li>Sample: </li></ul><ul><ul><li>N = 147 drivers at highway service area </li></ul></ul><ul><ul><li>N = 95 drivers at parking lot in front of supermarket </li></ul></ul><ul><ul><li>30 % female, mean age 47 year (sd = 16.5) </li></ul></ul>
  4. 4. Study 1: Results <ul><ul><li>Of sample, 56% use / own navigation systems </li></ul></ul><ul><ul><li>Of users, 63% use a mobile device, 36% a built-in navigation system </li></ul></ul><ul><ul><li>There are relevant differences between owners and non-owners: </li></ul></ul><ul><ul><ul><li>Owners of navigation systems have higher annual mileage </li></ul></ul></ul><ul><ul><ul><li>Owners of navigation systems drive more often on unfamiliar routes </li></ul></ul></ul><ul><ul><ul><li>Owners of navigation systems rate both system functions as more useful </li></ul></ul></ul>
  5. 5. Study 1: Relevance for the FOT <ul><li>Conduction of the FOT: </li></ul><ul><ul><li>To enhance the likelihood of system effects, a sample should be recruited with high mileage drivers </li></ul></ul><ul><ul><ul><li>Effect of navigation system probably mostly on unfamiliar trips </li></ul></ul></ul><ul><ul><ul><li>With higher mileage the likelihood of unfamiliar trips rises </li></ul></ul></ul><ul><ul><ul><li>Low mileage drivers only rarely experience the most relevant use case (unfamiliar route) </li></ul></ul></ul><ul><li>Analysis of the FOT-data (especially up-scaling of results for larger sample): </li></ul><ul><ul><li>Results from FOT-sample can not be transferred linearly to larger sample </li></ul></ul><ul><ul><li>For up-scaling, the likelihood of unfamiliar trips has to be considered </li></ul></ul>
  6. 6. Study 2: Aims of the experiment <ul><li>Part of pilot activities in euroFOT: </li></ul><ul><ul><li>Test of data logging </li></ul></ul><ul><ul><li>Implementation of data base </li></ul></ul><ul><ul><li>Implementation of data analysis </li></ul></ul><ul><ul><ul><li>Implementation and evaluation of algorithms for calculating derived measures (e.g. to detect certain events) </li></ul></ul></ul><ul><ul><li>Evaluation of planned data analysis </li></ul></ul><ul><ul><ul><li>Can the planned analysis differentiate between driving with and without navigation system? </li></ul></ul></ul>
  7. 7. Study 2: Experimental design <ul><li>Experiment with Daimler prototype vehicle for euroFOT </li></ul><ul><ul><li>Data logging is comparable to FOT data </li></ul></ul><ul><ul><li>Logging of CAN-data, GPS-position and video data </li></ul></ul><ul><li>Experimental design: </li></ul><ul><ul><li>In real traffic </li></ul></ul><ul><ul><li>Defined circular route with N=10 different destinations </li></ul></ul><ul><ul><li>Drivers are not familiar with the destinations </li></ul></ul><ul><ul><li>N = 10 drivers, 5 with navigation system and 5 without navigation system </li></ul></ul><ul><ul><li>Experimenter protocols relevant events (e.g. turnarounds) during the drive </li></ul></ul><ul><ul><li>Duration 4-5 hours </li></ul></ul>
  8. 8. Study 2: Route
  9. 9. Study 2: Implementing derived variables <ul><li>For analysis of FOT-data, event detection algorithms are needed </li></ul><ul><ul><li>Automated search for relevant events </li></ul></ul><ul><ul><li>If possible, no extensive control through video analysis necessary </li></ul></ul><ul><li>Example: Turnaround manoeuvre </li></ul><ul><ul><li>Manoeuvre, that directly relates to way finding performance </li></ul></ul><ul><ul><li>The experimenter protocolled turnarounds during the drive </li></ul></ul><ul><ul><li>In the CAN-data a turnaround is coded whenever reversing and steering wheel angle > 90° occur together within 60 seconds </li></ul></ul>
  10. 10. Study 2: Comparing protocols & event detection algorithm (Z=1.98, p<0.05) (Z=1.46 p=0.144)
  11. 11. Study 2: Analysis of errors in event detection <ul><li>Missed events (based on protocols): </li></ul><ul><ul><li>Turning e.g. at intersections where no reversing is necessary </li></ul></ul><ul><ul><li>Can not be differentiated based on CAN-data from turning off at intersections </li></ul></ul><ul><li>False positive events (based on analysis of video data): </li></ul><ul><ul><li>62 % of detected events are turnarounds </li></ul></ul><ul><ul><li>False positive are mostly parking manoeuvres (33%) </li></ul></ul>
  12. 12. Study 2: Improvement of algorithm <ul><li>The evaluation of detected events helps to find several weakness of the used algorithm. For example: </li></ul><ul><ul><li>Used algorithm was simple, but lead to events that endured about 60 seconds </li></ul></ul><ul><ul><ul><li>Coded time window is too long (in 71% of events turning at next intersection is included) </li></ul></ul></ul>
  13. 13. Study 2: Relevance for FOT <ul><li>Experimental approach has several advantages for implementing the analysis for an FOT: </li></ul><ul><ul><li>Because of the protocols, the algorithms for event detection can be evaluated regarding false positive and missed events </li></ul></ul><ul><ul><li>Weaknesses of the algorithms can be detected and improved </li></ul></ul><ul><ul><li>In case no improvement is possible, the impact of errors can be described for the data from the experiment </li></ul></ul><ul><ul><li>The planned analysis can be tested regarding its ability to detect effects of the tested system </li></ul></ul><ul><ul><li>The power of an purely automated analysis can be compared to an analysis relying on expert rating </li></ul></ul>
  14. 14. Conclusion <ul><li>It has been tried to combine the expensive approach of an FOT with less costly methods: </li></ul><ul><li>A final evaluation of the contribution of the pre-studies is not possible before the analysis of the FOT-data. </li></ul><ul><ul><li>Questionnaire survey provides information for a larger sample </li></ul></ul><ul><ul><li>Relevant for sample recruitment and planned up-scaling of results </li></ul></ul><ul><ul><li>Experiment provides well described and controlled data </li></ul></ul><ul><ul><li>Relevant for implementing and testing the analysis </li></ul></ul>
  15. 15. Thank you for your attention! For further information please contact [email_address] www.izvw.de

×