With growing usage of computer systems in daily life, a natural and intuitive Human Computer Interaction (HCI) method to support the embedding of computer systems in our environment seems necessary. Gestures are of utmost importance for design of natural user interfaces. Hand gesture recognition to extract meaningful expressions from the human hand movements and postures is being used for different applications. Yet, the recognition of hand gestures that contain different hand poses can be challenging. In this paper HANDY system is proposed for hand gesture recognition that is flexible to be trained to recognise a variety of user-defined gestures defined as sequences of static hand postures. The system has been designed to be used in uncontrolled environments, to handle dynamic and cluttered backgrounds, and without the need of using any wearable sensor or any specific clothing. Evaluation results show a good average accuracy in gesture recognition.

1. HANDY: A Configurable Gesture Recognition System
Mahsa Teimourikia1
, Hassan Saidinejad2,
,and Sara Comai3
Politecnico di Milano
1
mahsa.teimourikia@polimi.it, 2
hassan.saidinejad@polimi.it,
3
sara.comai@polimi.it March 2014

2. Polo Territoriale di Como
Motivations
2
• To embed computer systems in the environments
we need more natural ways of Human Computer
Interactions
• Gesture-based interfaces are one of the natural
ways that humans can use to interact with
computers
• Gesture-based interfaces are still in an infant stage.
And still not so natural!
• More fine grained gestures should be recognized
• Users should be able to define their preferred
gestures

3. Polo Territoriale di Como
Objectives
•Design and development of a gesture-based
interface, adaptable to a specific user
•Recognition of the dynamic gestures based
on the posture of the hand
3

4. Polo Territoriale di Como
Methodology
4

5. Polo Territoriale di Como
Hand Localization and Segmentation
5
The environment setting:
Hand Localization and segmentation:

6. Polo Territoriale di Como
Feature Extraction
6
Skeletonization: Using Voronoi Diagrams on Boundary Points

7. Polo Territoriale di Como
Hand Pose Estimation
• Resulted skeletons are not the same even in the same hand
poses
7
• Dynamic Time Warping (DTW) for pose estimation

8. Polo Territoriale di Como
Gesture Recognition
8

9. Polo Territoriale di Como
The Testing Application
9

10. Polo Territoriale di Como
Results
10
Average Recognition Accuracy
•Static Hand Pose Estimation : 83.53%
•Gesture Recognition: 95.57%
• 7 persons, ages between 24 to 60, 5 right handed and 2
tested the system with the left hand.

11. Polo Territoriale di Como
Conclusions
• This work has presented an approach for a flexible hand pose and
gesture recognition system that can be adapted to the special needs
of the users.
The interface:
• Is configurable for different hand poses
• Acceptable accuracy
• Can be trained with minimal effort to recognize user defined
gestures and sequence of poses
• Can be used in uncontrolled environments, with dynamic
background and low illumination. And the user does not need to use
any wearable devices.
11

12. Polo Territoriale di Como
Future Works
• As future work, hand pose estimator will be improved.
• Further extension of this system may include also facial
expression recognition and vocal.
• This work can be used in several applications such as
home automation.
• Since the proposed system is customizable it can be used
for HCI purposes for people with different disabilities or
needs.
14

13. Polo Territoriale di Como
Thank You
15