This document summarizes a study that validated a driving simulator by comparing driving performance between a simulated drive and an on-road drive. 47 participants drove a pre-defined route in both settings while an assessor measured errors. Several driving tasks showed no significant differences, but speed maintenance, lane positioning, and signaling showed significant differences. The study concluded the simulator can approximate some tasks but not all. Future research is needed to understand validation failures and improve simulator programming to better replicate real driving conditions.

1. CE 694 CreditSeminar
Validation Study of Driving
Simulator
Presented by
K.Raghupathi
153040008
Under supervision of
Prof. P. Vedagiri

2. Introduction
 Driving simulator a technical device which creates the
virtual-reality of driving.
 Important research tool in Transportation research
 Especially in road safety
 It allows safe and relevant environment to test driving
performance
2

3. Different Driving simulators
(https://www.carsim.com)
3

4. Objective of the study
 To have a good command of driving simulators
 To know its use in various fields
 Mainly focuses on validation
 A typical simulator necessarily needs validation
 A validation study of UC-win/Road driving simulator has
been discussed.
4

5. Background
 Need for the driving simulator:
• Human Factor/ Driver behavior studies
• Evaluation and Design of Transport/Road Infrastructures
• Education and training
 Types vary from fixed ones to moving based ones
 Advanced ones has high level applications like Engineering
development, ADAS, ITS etc.
5

6. Advanced Driving simulators
NADS-1 Driving simulator
(https://www.nads-sc.uiowa.edu)
VTI driving simulator IV
(www.vti.se/driving-simulators)
6

7. Literature review
 Validity : Extent to which results obtained are true.
 Simulator validation can have various approaches,
methodologies and criteria.
 Validation approach :
 Analytical approach
 Experimental approach
7

8.  Validation methodology:
 Define the validation approach
 Define the validation objectives
 Define the independent variables
 Define the dependent variables
 Define the type of statistical test
 Analysis of the results
 Validation Criteria
 Internal and external validity criteria
 Relative and absolute validity criteria
8

9. Outcome from literature review:
 Present study carried under analytical approach.
 Drivers are requested to perform driving in both
assessments.
 A statistical test is used for comparison.
 Internal and external criteria is considered as the primary
concern is driving behavior.
9

10. Case study
A validation study of driving errors using a driving simulator
Lynn Meuleners and Michelle Fraser
 A laboratory based driving simulator-UC-win/Road driving
stimulator is used
(http://www.forum8.co.jp)
10

11. Methodology
On-Road driving assessment:
 Study area : Stretch of road in Perth city, Western Australia
 Participants: 47 (26-male and 21-female, 18-69 years)
 Route consists :
 Two give way signs
 One stop sign, One merge
 Three uncontrolled intersections
 Five sets of traffic lights
 Five roundabouts
11

12. Driving simulator assessment:
 Route is replicated in simulator
 All traffic conditions except hazardous situations were
implemented
 Same participants were used
 Errors measured by driving assessor
 On road driving assessed by Occupational therapist
 15-20 minutes taken to complete
12

13. Images in driving simulator
corresponding to on road routes
(Lynn Meuleners and Michelle Fraser)
On-road driving route
(Lynn Meuleners and Michelle Fraser)
13

14. Performance measures
 Special driving assessment sheet
 Error measurements are
 Observation : Mirror checking
 Speed control : Correct speed for road
 Planning and judgment
 Vehicle positioning : Gap selection
 Physical control of vehicle : Indicators using
 Road laws : Obeying signs
14

15. Results and discussion:
 Statistical test used: paired sample t-tests
Simulator
Mean (SD)
On-road
Mean (SD)
p-value
Mirror checking 1.57 (1.15) 2.00 (1.42) 0.12
Left, right, forward observations 1.00 (1.42) 1.05 (1.44) 0.84
Correct speed for road 0.15 (0.67) 1.10 (1.45) <0.001
Maintaining speed 0.10 (0.55) 0.06 (0.01) 0.15
Speed at intersections 0.73 (1.30) 0.68 (1.26) 0.82
Gap selection 0.01 (0.03) 0.36 (0.99) 0.06
Maintaining lane positioning 0.57 (1.19) 0.15 (0.67) 0.02
Obeying stop signs 0.41 (0.01) 0.52 (0.39) 0.31
Obeying traffic lights 0.03 (0.01) 0.05 (0.39) 0.32
Indicating 2.26 (1.30) 1.73 (1.49) 0.04
Total errors mean 12.78 (6.30) 13.78 (6.98) 0.46
15

16. Conclusions of case study:
 Study can approximate specific driving tasks
• mirror checking
• speed at intersections
• left, right and forward observations
• maintaining speed and violation signs
 Significant difference in
• lane position maintenance
• gap selection
• indicating and driving the correct speed
16

17. Limitations:
 The sample size considered was small
 Two researchers of the study (ICC:0.6-0.8)
 Failing to provide immediate responses
 Some road characteristics are not replicated properly
 volunteer bias of the sample
17

18. Conclusions
 Proper validation methodology required to get accurate
outcomes of the study.
 Considerable amount of sample size to avoid statistical
limitations.
 Participants must be same for both assessments.
 programmer should replicate real driving conditions in the
simulator
 There is no exact driving behavior of the driver in simulators
18

19. Future scope
 Future research is needed to find validation failure causes
for some tasks.
 Proper programming is required to ensure at most
replication of real scenario.
19

20. References
 Allen, R.W., Mitchell, D.G., Stein, A.C. and Hogue, J.R. (1991). Validation of
real-time man in the loop simulation. VTI Report. No 372A, Part 4, pp. 18-31.
 Bella, F. (2008). Driving simulator for speed research on two-lane rural
roads. Accident Analysis & Prevention, 40(3), pp.1078-1087.
 Blaauw, G.J. (1982). Driving experience and task demands in simulator and
instrumented car: a validation study. Human Factors, 24(4), pp. 473-486.
 Blana, E. (1996). Driving Simulator Validation Studies: A Literature Review.
Institute of Transport Studies, University of Leeds, UK, Working Paper 480.
 Carsim.com, (2015). Mechanical Simulation Corporation. [online] Available at:
https://www.carsim.com [Accessed 17 Sep. 2015].
20

21.  Dorn, L. and Barker, D. (2005). The effects of driver training on simulated
driving performance. Accident Analysis & Prevention, 37(1), pp.63
 Godley, S., Triggs, T. and Fildes, B. (2002). Driving simulator validation for
speed research. Accident Analysis & Prevention, 34(5), pp.589-600.
 Meuleners, L. and Fraser, M. (2015). A validation study of driving errors using a
driving simulator. Transportation Research Part F: Traffic Psychology and
Behaviour, 29, pp.14-21.
 Nads-sc.uiowa.edu, (2015). The National Advanced Driving Simulator. [online]
Available at: https://www.nads-sc.uiowa.edu [Accessed 17 Nov. 2015].
 Statistics Solutions, (2015). Paired Sample T-Test - Statistics Solutions. [online]
Available at: http://www.statisticssolutions.com/manova-analysis-paired-
sample-t-test [Accessed 30 Oct. 2015].
21