The document presents a car driver skills assessment system using posture recognition to aid novice drivers by providing real-time feedback through an alarm system. It utilizes markerless sensors to accurately measure driving postures and ensure safety while learning. Experiments showed high accuracy rates in recognizing driving postures, indicating its effectiveness in indoor training environments.

1. Car Driver Skills
Assessment using
Posture Recognition
Presented By: sumit kadyan

2. Authors
• Madalina –Ioana Toma (Transilvania
University of Brasov)
• Leon J.M. Rothkrantz (Delft University of
technology)
• Csaba Antonya (M.I.Toma)

3. Need For it
• Difficult to learn driving in real life
scenario
• Safety issues
• Time boundation
• Learning driving as a Novice is a key in
driving career

4. Introduction
• Recognition of driving posture with High
Accuracy
• Feedback mechanism for novice drivers using
Alarm system.
• Experiment conducted in real time.

5. What Sets it apart?
• Recognition of complete body parts.
• Use of “Markerless” Sensors.
• Provides accurate measurement of joint
configuration and rapid movements of
hands.

6. Simulation Environment
framework
KINECT EYELINK2 CLIPS
LOGITECH
G27
TORCS PC

7. Pictorial Representation

8. Framework Components
• Kinects-sensing upper body movements
• Torcs-3D car simulator
• Clips-For rule based expert system
• Eyelink 2 device –for sensing eye and gaze
movement

9. Markerless Sensor

10. Markerless Sensor
• Uses pattern recognition principle
• Monitor process quality via control panel
or via Ethernet
• Reproducibility of 0.6 mm
• Plug can be rotated 90°
• High scanning speed of 7 m/s

11. KINECT

12. KINECT Sensor
• RGB camera sensor
• Configuration is done using Sdk tool by
windows
• IR Emitter and IR depth Sensor
• Used for tracking upper body movements

13. EYELINK II

14. Eye link 2
• High resolution and data rate
• Head mounted video-based eye tracker.
• Used for tracking eyes movement and
head orientation
• Two eye cameras allow binocular eye
tracking

15. CLIPS
• C Language Integrated Production System.
• CLIPS incorporates a complete object-
oriented language(COOL) for writing expert
systems.
• COOL combines the programming paradigms
of procedural, object oriented and logical
(theorem proving) languages.
• Provides High Portability.

16. Logitech G27
• Provides simulation Environment with TORCS.
• USB Interface.

17. TORCS virtual environment

18. TORCS
• 3D car simulator supporting input devices(
steering wheels, joystick, game pads etc.)
• Provides connection, configuration and
synchronization.
• Written in C++ and open source avaliable
under GPL license
• Easy to add/create content
• Excellent performance and stability

19. Related Work
• Pose Estimation
• Gaze Detection
• Focused only on Expert Drivers.
• Analyses done using offline techniques like
silhouettes, bounding boxes.

20. How it Works?
• Takes real time parameters from sensors
and environment.
• Refers to an expert rule based system to
determine the driving postures and give
feedback ,also sound an alarm if the
novice driver posture is wrong.
• Uses the clips inference engine
• Matching takes place between current
state of fact list and list of instances

21. Description of the rules

22. Defining Rules
• Rules for recognizing driving postures are
stored in the knowledge base system.
• Rules for driving posture:
 DP1,DP2,DP3,DP4
 DP1-Left hand postures
 DP2-Right hand postures
 DP3-Eye and Head postures
 DP4-leg postures

23. Working
• Each group represnts a postuers runsin
paralles with the other
• A driver posture is represnted a key poses
• Which is a combination of 2- 5 key poses
• These are the inputs to the CLIPS
• In a driving task the driving posture used
to perform that maneuver are defined in a
specific order

24. Finite state machine
diagram

25. DFSM
• Determisnntic finite state machine
• , S, s0 , , F
• -Input alphabet(from the sensors)
• S-Finite set of states (showing transition in
DP1 ,DP2 …DP4
• s0- Initial state(When the system is calibrated
for start )
• -state transition(from one
• F-final state

26. Diagram of the interface

27. Experiment
• Experiment was focused on developing a
assistive intelligent system for indoor training
of novice drivers
• Experiments conducted in laboratory with
proper lighting for sensors
• 2 kind of experiments
• One for robustness and performance of
posture recognition the novice driver without
traffic
• 2 in is the complete framework evaluation.

28. Two Experiments
Experiments
Conducted
Detecting Robustness
and accuracy of
posture recognition
for novice drivers
Complete framework
evaluation and
provide feedback

29. Participants
• 12 participants
• 8 males and 4 females
• All having driver license
• With little or no experience

30. Results of Experiment 1
• Every subject performed the postures for
10 times
• Driving postures recognition rate achieves
96.4% accuracy
• Driving posture stability achieves 96.21%
accuracy
• GOOD” and “WORST” messages

31. Table of Results

32. Experiment 2 : Rules
• driver needs to start the car (StC)
• driver wants to drive away (DA)
• driver keeps the lane (KL)
• driver increases the speed (IS) or decreases
the speed (DS) based on traffic signs
• driver wants to take over (TO) or change lane
(CL)
• driver wants to make a forward parking (FP)
driver wants to stop the car (SpC).

33. Results of Experiment 2
• In the StC situation we achieved 88%
correct postures detectioni
• In the IS and DS speed variation situations
we achieved an accuracy of 100%.
• A lower accuracy of less than 70% we
obtained in TO and FP

34. Results experiment 2
• In the StC situation we achieved 88%
correct postures detection.
• In the IS and DS speed variation situations
we achieved an accuracy of 100%.
• A lower accuracy of less than 70% we
obtained in TO and FP

35. Table of Results

36. Conclusion
• To improve the take over and forward
parking by combining probabilistic
methods reducing uncertainty of certain
driver postures.

37. References
• Toma, Madalina-Ioana; Rothkrantz, Leon J.M.; Antonya, Csaba, "Car driver
skills assessment based on driving postures recognition," Cognitive
Infocommunications (CogInfoCom), 2012 IEEE 3rd International Conference
on , vol., no., pp.439,446, 2-5 Dec. 2012
• I. Lefter, L.J.M. Rothkrantz, P. Bouchner, P. Wiggers: “A multimodal car driver
surveillance system in a military area”, Driver Car Interaction & Interface,
2010.
• Y.F. Lu, and Ch.Li: “Recognition of Driver Turn Behavior Based on Video
Analysis”, Journal of Advanced Materials Research Vol. 433-44, pp 6230-
6234, 2012.
• D.B. Kaber, Y. Liang, Y. Zhang, M. L. Rogers, and S. Gangakhedkar: “Driver
performance effects of simultaneous visual and cognitive distraction and
adaptation behavior”, Journal of Transportation Research Part F 15, pp. 491–
501, 2012.