Samantha Jamson, University of Leeds


Published on

Methodological challenges and solutions in euroFOT

Published in: Automotive, Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • The first European Large-Scale Field Operational Test on In-Vehicle Systems OR: First European large-scale field operational test for evaluating the impact of nowadays active safety systems in vehicles. an FP7 Integrated Project in the frame of the DG Information-Society Programme
  • Samantha Jamson, University of Leeds

    1. 1. Methodological challenges and solutions in euroFOT Samantha Jamson September 2009 University of Leeds
    2. 2. euroFOT Objectives <ul><li>Apply a common European approach for Field Operational Tests </li></ul><ul><li>Perform multiple coordinated tests of Intelligent Vehicle Systems with ordinary drivers in real traffic </li></ul><ul><li>Investigate performance, driver behaviour and user acceptance </li></ul><ul><li>Assess the impacts on safety, efficiency and the environment, </li></ul>Test Management Centres
    3. 3. Functions <ul><li>Longitudinal control functions </li></ul><ul><ul><li>FCW Forward Collision Warning </li></ul></ul><ul><ul><li>ACC Adaptive Cruise Control </li></ul></ul><ul><ul><li>SL Speed Limiter </li></ul></ul><ul><li>Lateral control functions </li></ul><ul><ul><li>BLIS Blind Spot Information System </li></ul></ul><ul><ul><li>LDW Lane Departure Warning </li></ul></ul><ul><ul><li>IW Impairment Warning </li></ul></ul><ul><li>Others </li></ul><ul><ul><li>CSW Curve Speed Warning </li></ul></ul><ul><ul><li>FEA Fuel Efficiency Advisor </li></ul></ul><ul><ul><li>SafeHMI Safe Human Machine Interaction </li></ul></ul>
    4. 4. FESTA Function Identification and Description Performance Indicators Use Cases Research Questions and Hypotheses Measures and Sensors Study Design Research Question and Hypotheses Analysis Socio - Economic Impact Assessment System and Function Analysis Ethical and Legal Issues I m p l e m e n t a t i o n P l a n Database Performance Indicators Measures Data Decoding Data Acquisition Data Analysis The FOT Chain
    5. 5. From Research Questions to Hypotheses #I <ul><li>Research Questions (Example) </li></ul><ul><ul><li>ACC: Does ACC improve the drivers’ distance keeping behaviour? </li></ul></ul><ul><li>Research questions are the reason for conducting the study. </li></ul><ul><li>They are often rather general. </li></ul><ul><li>More detail is required to plan the study well </li></ul><ul><li> formulate hypotheses … </li></ul>
    6. 6. From Research Questions to Hypotheses #2 <ul><li>Hypotheses (Examples) </li></ul><ul><ul><li>ACC: The percentage of TTC(min) decreases by at least 20 per cent. </li></ul></ul><ul><ul><li>ACC: The variance in headway in a following situation decreases. </li></ul></ul><ul><li>Hypotheses must be testable. </li></ul><ul><li>They are specific, usually directional and identify the Performance Indicator(s) (PI) in question. </li></ul>
    7. 7. From Research Questions to Hypotheses #3 <ul><li>From PIs to Sensors </li></ul><ul><ul><li>Percentage of TTC(min): Necessary to collect speed, distance to vehicle ahead, speed of vehicle ahead. </li></ul></ul><ul><ul><li>Variance in headway: Necessary to collect distance to vehicle ahead. </li></ul></ul><ul><li>PI determine which measures are necessary. </li></ul><ul><li>The measures have to be logged via sensors . </li></ul>
    8. 8. Procedure for euroFOT <ul><li>The research question has been asked. </li></ul><ul><li>The research question has to be broken down into hypotheses (specific, testable, unidirectional). This is a vital step!!! </li></ul><ul><li>A hypothesis contains at least one PI. </li></ul><ul><li>The PI is looked up in the PI matrix, which informs about the measures necessary for the PI. </li></ul><ul><li>The necessary measures are looked up in the measures matrix, which informs about the sensors that can be used to log the measures. </li></ul><ul><li>(PI & measures matrix started within FESTA) </li></ul>
    9. 9. Defining the experimental framework <ul><li>Top-down procedure, setting out various recommendations & assumptions. </li></ul><ul><li>Face-to-face clinics helped us to brainstorm and use the expertise of the consortium </li></ul><ul><li>Produce a design that would enable the collection and analysis of data that could be used to produce statistically reliable results. </li></ul>
    10. 10. Identifying the constraints <ul><li>Budget – the euroFOT workplan and budget are, as in all research projects, the major deciding factor when developing the experimental procedures. </li></ul><ul><li>Vehicles – the types of vehicles will depend largely on the availability of the functions on a particular model. Higher class vehicles which inevitably have more functions available appeal to older individuals with more disposable income than say a younger driver who may favour the lower insurance premiums and running costs of a smaller vehicle. </li></ul><ul><li>Economic climate – reduced vehicle sales. European new passenger car sales fell 7.8 percent in 2008, the high-end markets suffering most as drivers may opt for smaller, cheaper alternatives. This squeezes the available participants. </li></ul><ul><li>Drivers – drivers will mostly be self-selected (with the exception of the truck drivers who are selected by the fleet manager). Bias will be noted and compared to the country specific population statistics </li></ul><ul><li>Multiple functions – the vehicles are models which are manufactured for the current market. They are, in most cases, not specific “test” vehicles but are owned or leased by the driver. In this respect the eight functions under consideration in euroFOT are often sold in bundles – groups of functions. </li></ul>
    11. 11. Power analysis <ul><li>Power analyses are undertaken to determine the required sample size in order to make accurate and reliable statistical conclusions, and to determine how likely the statistical test will be able to detect effects of a given size in a particular situation. </li></ul><ul><li>When planning a FOT three interrelated questions arise regarding the experimental design: </li></ul><ul><li>how large is the effect of the system under investigation, </li></ul><ul><li>how many cars have to be equipped to find this effect and </li></ul><ul><li>how long do they have to be driven to obtain a sufficient amount of data. </li></ul>
    12. 12. Power analysis conclusions <ul><li>Monte Carlo simulations have shown that when at least 120 participants are included, with a mileage of 15k per year, even with the small effect size that can be expected in a FOT, a sufficient power will be reached. </li></ul><ul><li>Including more cars or more unique participants should take precedence over measuring for longer periods: e.g. measuring for a year with 60 participants could fail where measuring for half a year with 120 participants could provide a significant result. </li></ul><ul><li>Reduce the variance measured between participants - choose a homogenous group of drivers for example male drivers between 30-40 years of age with similar mileage. This would improve the power, but at the cost of the generalisability (external validity) of the results. </li></ul><ul><li>Baseline and treatment phase should be as near to equal as possible </li></ul>
    13. 13. Harmonised experimental designs
    14. 14. Harmonised questionnaires <ul><li>All VMCs are using the same core questionnaires (same scales, items etc) with flexibility for additional items </li></ul>
    15. 15. How has this procedure helped euroFOT? The approach taken in this workpackage, has, as far as possible given the identified constraints, enabled the VMCs within euroFOT to develop experimental procedures that are both robust and unified With regards to the experimental designs , we have been able to achieve a coordinated effort in realising a common experimental with only a couple of deviations We have also been successful with regards to developing the questionnaires that provide the subjective data in a unified format for use in the impact analysis at the end of the project.
    16. 16. Questions? <ul><li>Dr Samantha Jamson </li></ul><ul><li>[email_address] </li></ul>
    17. 17. October A. Etemad – XYZ Workshop