Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Planning of a field operational test on navigation systems:   Implementation and evaluation of pre-studies providing backg...
Background <ul><li>Presented research is part of the EC-funded project euroFOT: </li></ul><ul><ul><li>In 2010 a field oper...
Study 1: Questionnaire survey <ul><li>Development of short questionnaire (5-10 minutes) </li></ul><ul><ul><li>Usage and ev...
Study 1: Results <ul><ul><li>Of sample, 56% use / own navigation systems </li></ul></ul><ul><ul><li>Of users, 63% use a mo...
Study 1: Relevance for the FOT <ul><li>Conduction of the FOT: </li></ul><ul><ul><li>To enhance the likelihood of system ef...
Study 2: Aims of the experiment <ul><li>Part of pilot activities in euroFOT: </li></ul><ul><ul><li>Test of data logging </...
Study 2: Experimental design <ul><li>Experiment with Daimler prototype vehicle for euroFOT </li></ul><ul><ul><li>Data logg...
Study 2: Route
Study 2: Implementing derived variables <ul><li>For analysis of FOT-data, event detection algorithms are needed </li></ul>...
Study 2: Comparing protocols & event detection algorithm (Z=1.98, p<0.05)   (Z=1.46 p=0.144)
Study 2: Analysis of  errors in event detection <ul><li>Missed events (based on protocols): </li></ul><ul><ul><li>Turning ...
Study 2: Improvement of algorithm <ul><li>The evaluation of detected events helps to find several weakness of the used alg...
Study 2: Relevance for FOT <ul><li>Experimental approach has several advantages for implementing the analysis for an FOT: ...
Conclusion <ul><li>It has been tried to combine the expensive approach of an FOT with less costly methods: </li></ul><ul><...
Thank you for your attention! For further information please contact [email_address] www.izvw.de
Upcoming SlideShare
Loading in …5
×

Planning of a field operational test on navigation systems: Implementation and evaluation of pre-studies providing background information for the data analysis, Barbara Metz, Stefanie Schoch, Mariana Rakic & Friedemann Kuhn

1,392 views

Published on

  • Login to see the comments

  • Be the first to like this

Planning of a field operational test on navigation systems: Implementation and evaluation of pre-studies providing background information for the data analysis, Barbara Metz, Stefanie Schoch, Mariana Rakic & Friedemann Kuhn

  1. 1. Planning of a field operational test on navigation systems: Implementation and evaluation of pre-studies providing background information for the data analysis Barbara Metz, Stefanie Schoch, Mariana Rakic & Friedemann Kuhn
  2. 2. Background <ul><li>Presented research is part of the EC-funded project euroFOT: </li></ul><ul><ul><li>In 2010 a field operational test will take place with the aim to evaluate the effect and usage of navigation systems </li></ul></ul><ul><ul><li>Driving without navigation system, with navigation systems provided by BMW & Daimler and with a mobile navigation system will be compared </li></ul></ul><ul><li>General aim of euroFOT: Assess impact of investigated driver assistance systems on driving on a EU-level for a representative sample. </li></ul><ul><li>Two pre-studies will be presented which provide background information for planning and implementing the FOT-data analysis. </li></ul>
  3. 3. Study 1: Questionnaire survey <ul><li>Development of short questionnaire (5-10 minutes) </li></ul><ul><ul><li>Usage and evaluation of navigation system </li></ul></ul><ul><ul><li>Frequency of driving & unfamiliar routes </li></ul></ul><ul><ul><li>Other driving relevant background information </li></ul></ul><ul><li>Conduction Nov / Dec 2008 and June / July 2009 </li></ul><ul><li>Sample: </li></ul><ul><ul><li>N = 147 drivers at highway service area </li></ul></ul><ul><ul><li>N = 95 drivers at parking lot in front of supermarket </li></ul></ul><ul><ul><li>30 % female, mean age 47 year (sd = 16.5) </li></ul></ul>
  4. 4. Study 1: Results <ul><ul><li>Of sample, 56% use / own navigation systems </li></ul></ul><ul><ul><li>Of users, 63% use a mobile device, 36% a built-in navigation system </li></ul></ul><ul><ul><li>There are relevant differences between owners and non-owners: </li></ul></ul><ul><ul><ul><li>Owners of navigation systems have higher annual mileage </li></ul></ul></ul><ul><ul><ul><li>Owners of navigation systems drive more often on unfamiliar routes </li></ul></ul></ul><ul><ul><ul><li>Owners of navigation systems rate both system functions as more useful </li></ul></ul></ul>
  5. 5. Study 1: Relevance for the FOT <ul><li>Conduction of the FOT: </li></ul><ul><ul><li>To enhance the likelihood of system effects, a sample should be recruited with high mileage drivers </li></ul></ul><ul><ul><ul><li>Effect of navigation system probably mostly on unfamiliar trips </li></ul></ul></ul><ul><ul><ul><li>With higher mileage the likelihood of unfamiliar trips rises </li></ul></ul></ul><ul><ul><ul><li>Low mileage drivers only rarely experience the most relevant use case (unfamiliar route) </li></ul></ul></ul><ul><li>Analysis of the FOT-data (especially up-scaling of results for larger sample): </li></ul><ul><ul><li>Results from FOT-sample can not be transferred linearly to larger sample </li></ul></ul><ul><ul><li>For up-scaling, the likelihood of unfamiliar trips has to be considered </li></ul></ul>
  6. 6. Study 2: Aims of the experiment <ul><li>Part of pilot activities in euroFOT: </li></ul><ul><ul><li>Test of data logging </li></ul></ul><ul><ul><li>Implementation of data base </li></ul></ul><ul><ul><li>Implementation of data analysis </li></ul></ul><ul><ul><ul><li>Implementation and evaluation of algorithms for calculating derived measures (e.g. to detect certain events) </li></ul></ul></ul><ul><ul><li>Evaluation of planned data analysis </li></ul></ul><ul><ul><ul><li>Can the planned analysis differentiate between driving with and without navigation system? </li></ul></ul></ul>
  7. 7. Study 2: Experimental design <ul><li>Experiment with Daimler prototype vehicle for euroFOT </li></ul><ul><ul><li>Data logging is comparable to FOT data </li></ul></ul><ul><ul><li>Logging of CAN-data, GPS-position and video data </li></ul></ul><ul><li>Experimental design: </li></ul><ul><ul><li>In real traffic </li></ul></ul><ul><ul><li>Defined circular route with N=10 different destinations </li></ul></ul><ul><ul><li>Drivers are not familiar with the destinations </li></ul></ul><ul><ul><li>N = 10 drivers, 5 with navigation system and 5 without navigation system </li></ul></ul><ul><ul><li>Experimenter protocols relevant events (e.g. turnarounds) during the drive </li></ul></ul><ul><ul><li>Duration 4-5 hours </li></ul></ul>
  8. 8. Study 2: Route
  9. 9. Study 2: Implementing derived variables <ul><li>For analysis of FOT-data, event detection algorithms are needed </li></ul><ul><ul><li>Automated search for relevant events </li></ul></ul><ul><ul><li>If possible, no extensive control through video analysis necessary </li></ul></ul><ul><li>Example: Turnaround manoeuvre </li></ul><ul><ul><li>Manoeuvre, that directly relates to way finding performance </li></ul></ul><ul><ul><li>The experimenter protocolled turnarounds during the drive </li></ul></ul><ul><ul><li>In the CAN-data a turnaround is coded whenever reversing and steering wheel angle > 90° occur together within 60 seconds </li></ul></ul>
  10. 10. Study 2: Comparing protocols & event detection algorithm (Z=1.98, p<0.05) (Z=1.46 p=0.144)
  11. 11. Study 2: Analysis of errors in event detection <ul><li>Missed events (based on protocols): </li></ul><ul><ul><li>Turning e.g. at intersections where no reversing is necessary </li></ul></ul><ul><ul><li>Can not be differentiated based on CAN-data from turning off at intersections </li></ul></ul><ul><li>False positive events (based on analysis of video data): </li></ul><ul><ul><li>62 % of detected events are turnarounds </li></ul></ul><ul><ul><li>False positive are mostly parking manoeuvres (33%) </li></ul></ul>
  12. 12. Study 2: Improvement of algorithm <ul><li>The evaluation of detected events helps to find several weakness of the used algorithm. For example: </li></ul><ul><ul><li>Used algorithm was simple, but lead to events that endured about 60 seconds </li></ul></ul><ul><ul><ul><li>Coded time window is too long (in 71% of events turning at next intersection is included) </li></ul></ul></ul>
  13. 13. Study 2: Relevance for FOT <ul><li>Experimental approach has several advantages for implementing the analysis for an FOT: </li></ul><ul><ul><li>Because of the protocols, the algorithms for event detection can be evaluated regarding false positive and missed events </li></ul></ul><ul><ul><li>Weaknesses of the algorithms can be detected and improved </li></ul></ul><ul><ul><li>In case no improvement is possible, the impact of errors can be described for the data from the experiment </li></ul></ul><ul><ul><li>The planned analysis can be tested regarding its ability to detect effects of the tested system </li></ul></ul><ul><ul><li>The power of an purely automated analysis can be compared to an analysis relying on expert rating </li></ul></ul>
  14. 14. Conclusion <ul><li>It has been tried to combine the expensive approach of an FOT with less costly methods: </li></ul><ul><li>A final evaluation of the contribution of the pre-studies is not possible before the analysis of the FOT-data. </li></ul><ul><ul><li>Questionnaire survey provides information for a larger sample </li></ul></ul><ul><ul><li>Relevant for sample recruitment and planned up-scaling of results </li></ul></ul><ul><ul><li>Experiment provides well described and controlled data </li></ul></ul><ul><ul><li>Relevant for implementing and testing the analysis </li></ul></ul>
  15. 15. Thank you for your attention! For further information please contact [email_address] www.izvw.de

×