Gesture recognition technology with its applications.

1. SEMINAR (CS-307)
Presentation on:
Gesture Recognition Technology
SUBMITTED TO:
Prof. Pragya Jain
CS/IT Department
SUBMITTED BY:
Aishwarya Bharadwaj
0903CS121008
MAHARANA PRATAP COLLEGE OF
TECHNOLOGY

3. Introduction
 Gesture recognition is a topic in Computer Science and
language technology with the goal of interpreting
human gestures via mathematical Algorithms.
 It can be seen as a way for computers to begin to
understand human body language, thus building a
richer bridge between machines and humans than
primitive text user interfaces or even GUIs , which still
limit the majority of input to keyboards and mouse.

4. Contents
1. Gesture Types
2. Uses
3. Input Devices
4. Algorithms
5. 3-D Model based algorithms
6. Skeletal based algorithms
7. Appearance based algorithms
8. Disadvantages

5. Gesture Types
 In computer interfaces, two types of gestures are
distinguished :
 Online gestures: We consider online gestures, which
can also be regarded as direct manipulation gestures
They are used to scale or rotate a tangible object.
 Offline gestures: Those gestures that are processed
after the user interaction with the object. e. g. a circle
is drawn to activate a context menu.

6. Uses
 Sign Language Recognition
 For socially assistive robotics
 Directional indication through pointing
 Alternative computer interfaces
 Immersive Game technology
 Virtual Controllers
 Affective Computing
 Remote control

7. Input Devices
 Wired Gloves: They can detect finger bending with a
high degree of accuracy (5-10 degrees).The first hand-
tracking device was Data Glove. Uses optical wires.
 Depth-aware Cameras: One can generate a depth
map of what is being seen at a short range.
 Stereo Cameras: Using two cameras whose relations
to one another are known, we can get a 3-D
representation
 Controller-based gestures: These controllers act as
an extension of the body so that when gestures are
performed, some of their motion can be captured by
the software.

8. Algorithms

9. 3-D Model based Algorithms
• A more interesting approach would be to map simple
primitive objects to the person’s most important body parts
( for example cylinders for the arms and neck, sphere for
the head) and analyse the way these interact with each
other. Furthermore, some abstract structures like super-
quadrics and generalised cylinders may be even more
suitable for approximating the body parts.
• A real hand (left) is interpreted as a collection of vertices
and lines in the 3D mesh version (right), and the software
uses their relative position and interaction in order to infer
the gesture.

10. Skeletal based Algorithms
 Instead of using intensive processing of the 3D models and
dealing with a lot of parameters, one can just use a
simplified version of joint angle parameters along with
segment lengths. This is known as a skeletal
representation of the body, where a virtual skeleton of the
person is computed and parts of the body are mapped to
certain segments.
 The skeletal version (right) is effectively modelling the
hand (left). This has fewer parameters than the volumetric
version and it's easier to compute, making it suitable for
real-time gesture analysis systems

11. Appearance- based Algorithms
 These models don’t use a spatial representation of the body anymore,
because they derive the parameters directly from the images or videos using
a template database. Some are based on the deformable 2D templates of
the human parts of the body, particularly hands. Deformable templates are
sets of points on the outline of an object, used as interpolation nodes for
the object’s outline approximation. These template-based models are mostly
used for hand-tracking, but could also be of use for simple gesture
classification.
 Second approach is image sequences . Parameters for this method are
either the images themselves, or certain features derived from these.
• These binary silhouette(left) or contour(right) images represent typical
input for appearance-based algorithms. They are compared with different
hand templates and if they match, the correspondent gesture is inferred.

12. Disadvantage
 There are many challenges associated with the
accuracy and usefulness of Gesture Recognition
Software.
 For image-based gesture recognition there are
limitations on the equipment used.
 Items in the background or distinct features of the
users may make recognition more difficult.
 The distance from the camera and camera’s resolution
and quality also cause variations in recognition
accuracy.
 Gorilla arm: In periods of prolonged use, users’ arm
began to feel discomfort.

14. Touchable wind energy
 GE has been active in
Germany for over 100 yrs.
 For the first time ever the
innovative GRT was used
to translate the movements
of the participants’ arms
into moving virtual wind
turbines on a multiple HD
video wall.
 The promoters invited
travelers at Frankfurt
Airport to generate virtual
wind energy.

15. Interactive Floors and Walls!!
 GestureFX is one of the
world’s most advanced GRT
for motion-control
interactive displays, signs
and surfaces.
 It’s patented body tracking
technology responds to
body movement to project
dynamic interactive
multimedia content, special
effects, interactive
advertising or games onto
any surface.

16. Virtual Guitar!!
 The user’s actions are
read by the input device,
such as a webcam and
passed through gesture
recognition.
 A musical intelligence
module interprets these
gestures and sends
commands to the sound
model, which produces
final sound

17. CONCLUSION
 Improvements and Future:
 Gesture recognition algorithm is relatively durable and
accurate.
 Convolution can be slow, so there is tradeoff between
speed and accuracy.
 This computing is not only going to reduce the
hardware impact of the system but also it increases the
range of usage of physical world object instead of
digital objects like keyboards, mouse.

18. THANK YOU