1. WELCOME

2. MEMS ACCELEROMETER BASED
NONSPECIFIC – USER HAND
GESTURE RECOGNITION
GRACE ABRAHAM
ROLL . NO : 25
S7 ECE
SNMIMT.
26 /7/ 2013 Dept. of ECE 2

3. CONTENTS
• INTRODUCTION
• GESTURE MOTION ANALYSIS
• SENSING SYSTEM
• SYSTEM WORK FLOW
• GESTURE SEGMENTATION
• MODEL 1 : BASED ON SIGN SEQUENCE AND HOPFIELD NETWORK.
• MODEL 2 : BASED ON VELOCITY INCREMENT
• MODEL 3 : BASED ON SIGN SEQUENCE AND TEMPLATE MATCHING.
• EXPERIMENTAL RESULTS
• ADVANTAGES
• APPLICATIONS
• CONCLUSION
26/7/2013 Dept. of ECE 3
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

4. INTRODUCTION
• Human – Machine Interactions
• Physical Gestures
• Gesture Recognition
 7 Hand Gestures
 MEMS Accelerometer
 A Micro Inertial Measurement Unit ( IMU)
 2 Methods
 Approaches
o Vision – Based ( Limitation )
o Accelerometer Based
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition
o Template Matching
o Statistical Matching
26/7/2013 Dept. of ECE 4

5. • 3 Gesture Recognition Models
• 7 Gestures
• Inputted to MEMS 3 – Accelerometer
• Gesture Segmentation Algorithm
• 100’s of data to 8 number code
• Gesture Recognition
 Sign Sequence & Hopfield Based
 Velocity Increment Based
 Sign Sequence & Template Matching Based
 Up, down, left, right, tick, circle, cross
26/7/2013 Dept. of ECE 5
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

6. BLOCK DIAGRAM
SENSING CHIP
DATA
PROCESSING
GESTURE
SEGMENTATION
GESTURE
RECOGNITION
GESTURE INPUT
RECOGNISED GESTURE
OUTPUT
26/7/2013 Dept. of ECE 6
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

7. GESTURE MOTION ANALYSIS
26/7/2013 Dept. of ECE 7
Fig 1 : Coordinate System
Fig 2 : Gesture up motion decomposition
• Motion in vertical plane ( x – z plane)
• Accelerations on x-z plane
• Up Gesture
 Up Gesture
 Circle Gesture
o X axis : no acceleration
o Z axis : negative – positive – negative
o X axis : positive – negative – positive
o Z axis : negative – positive – negative - positive
 Velocity zero at pt. 1 & 2
 Sign changes at pt. 3 & 4
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

8. Fig 3 : Predicted velocity and acceleration in
the z-axis
Fig 4 : Real acceleration plot
• One Axis – up & down, left & right
• Two Axis – tick, circle, cross (complex)
• Acceleration changes in z axis
• Real acceleration is the same with the
prediction
• Unique acceleration pattern
 1 to 3 :-ve; V changes from 0 to max. at
3
 3 to 4 :+ve; V changes from -ve to +ve
& max. at pt 4
 4 to 1 :-ve; V changes from +ve to zero
26/7/2013 Dept. of ECE 8
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

9. SENSING SYSTEM
Fig 5 : Sensing System
• MEMS 3 – axes acceleration
sensing chip
• Data management chip
• Bluetooth Wireless data chip
26/7/2013 Dept. of ECE 9
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

10. • MEMS ???
• ACCELEROMETER ???
 Micro Electro-Mechanical Systems
 Combination of mechanical functions & electronical functions
on same chip
 Micro fabrication technology
 Electromechanical device
 Measure acceleration forces
26/7/2013 Dept. of ECE 10
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

11. 26/7/2013 Dept. of ECE 11
SYSTEM WORK FLOW
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

12. Fig 6 : Motions of seven gestures
GESTURE SEGMENTATION
• DATA ACQUISITION
Horizontally place sensing device
Time interval not less than 0.2sec
Perform Gestures as shown in Fig 6
26/7/2013 Dept. of ECE 12
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

13. • GESTURE SEGMENTATION
 DATA PREPROCESSING
 2 Processes
o Remove vertical axis offsets by subtracting data points from mean
value
o Filter to eliminate noise
26/7/2013 Dept. of ECE 13
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

14. 26/7/2013 Dept. of ECE 14
Fig. 3.2. Segmentation of a seven-gesture sequence in the order up-down-left-right-tick-circle-cross.Fig. 3.2. Segmentation of a seven-gesture sequence in the order up-down-left-right-tick-circle-cross.Fig. 3.2. Segmentation of a seven-gesture sequence in the order up-down-left-right-tick-circle-cross.Fig. 3.2. Segmentation of a seven-gesture sequence in the order up-down-left-right-tick-circle-cross.
Fig 7 : Segmentation of a seven-gesture
sequence in the order up-down-left-right-tick-
circle-cross.
 SEGMENTATION
o Find terminal points
o We need
o 2 x n matrices generated
o Compare max. acceleration b/w terminal points with its mean value
o No. of columns = No. of gestures
 Amplitude of points
 Point separation
 Mean value
 Distance from nearest intersection
 Sign variation b/w 2 successive points
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

15. MODEL ONE : GESTURE
RECOGNITION BASED ON SIGN
SEQUENCE AND HOPFIELD
NETWORK
26/7/2013 Dept. of ECE 15
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

16. Fig. 4.1 . Sign sequence generationFig. 4.1 . Sign sequence generationFig. 4.1 . Sign sequence generationFig. 4.1 . Sign sequence generation
Fig 8 : Sign sequence generation
GESTURE RECOGNITION
• FEATURE EXTRACTION
 Examine the sign of the first mean point
of a gesture
 Store in gesture code
 Detect no. of sign changes
 Store the alternate signs in sequence in
the gesture code
 Code for the gesture in fig 8 is 1, -1, 1, -1
26/7/2013 Dept. of ECE 16
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

17. • GESTURE ENCODING
 Max. no. of signs for 1 gesture on 1 axis is 4
 Eight numbers in one gesture code
 Hopfield network can take only 1 & -1 as inputs
 +ve, -ve sign and zero are encoded as “1 1”, “-1 -1” and “1 -1”
 Each gesture has a unique 16 - number code
26/7/2013 Dept. of ECE 17
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

18. • HOPFIELD NETWORK AS ASSOCIATIVE MEMORY
 Recovery mechanism
 Weight matrix is constructed
sp - Pattern to be stored
P - Number of patterns
I - Identity matrix

 npTPp
p
P
sPIssw 1,1,)(1
  
q
sv )0(
qqTpp
p
p
Psssswvu  


)()0()1(
1
1
)1()(  nwvnu))(sgn()1( nuv 
))(sgn()( nunv 
,
outputnv )(
26/7/2013 Dept. of ECE 18
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

19. • GESTURE COMPARISON
 Gesture code is compared with the standard gesture codes
 Difference b/w the two codes is calculated
 Smallest difference indicates the most likely gesture
26/7/2013 Dept. of ECE 19
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

20. Table 1: Standard patterns for the seven gestures
26/7/2013 Dept. of ECE 20
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

21. MODEL TWO: GESTURE RECOGNITION
BASED ON VELOCITY INCREMENT
• Model deals with complex gestures
• Area bounded by acceleration curve & x axis
• Partitioned areas with alternate signs
• Normalization of area sequence,
- Normalized area
- Original area, - Max. area
maxA
A
A
original
norm 
 Increase/decrease in velocity
normA
originalA maxA
26/7/2013 Dept. of ECE 21
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition
Fig. 9 : Acceleration partition

22. • To avoid misalignment due to noise
• Compare velocity increment
– Two area sequences compared
– Comparison result
• Gesture with min. Value recognized
 Imagining curve has mass
 Obtain center of mass
 Two curves are aligned to coincide their centers of masses
 Subtracting 2 area sequence vectors
 
 
nnd
nn
nn
AAAAAAA
AAAAAS
AAAAAS





'
2
'
21
'
1
''
1
'
3
'
2
'
12
1........3211
.....
......
,,
21,ss
dA
25/7/2013 Dept. of ECE 22
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

23. 25/7/2013 Dept. of ECE 23
WORK FLOW CHART
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

24. MODEL 3:GESTURE RECOGNITION
BASED ON SIGN SEQUENCE &
TEMPLATE MATCHING
• Similar to model one
• No encoding of sign
sequence in to combinations
of -1s & 1s
• Not limited to specific users
Table 2: Gesture codes for model 3
25/7/2013 Dept. of ECE 24
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

25. EXPERIMENTAL RESULTS
• ACCURACY
Model III > I > II
• PERFORMANCE
Model III > I > II
• Model III has an overall
mean accuracy of 95.6%
Table 7.1.Comparison of gesture recognition accuracy(%) of three models
Table 3 : Comparison of gesture recognition
accuracy(%) of 3 models
25/7/2013 Dept. of ECE 25
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

26. ADVANTAGES
 User friendly
 Gesture patterns are not critical
 Noise filtering is not required
 User doesn’t require any advanced training
 Low power, compact and robust sensing
25/7/2013 Dept. of ECE 26
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

27. APPLICATIONS
 Character recognition in 3-D space
 To control a tv set
 Virtual keyboard
 Immersive game technology
 For socially assistive robotics
25/7/2013 Dept. of ECE 27
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

28. CONCLUSION
 Sensor data collection, segmentation & recognition
 Sign sequence of gesture is extracted
 100’s of data to code of 8 numbers
 Code compared with standard patterns
25/7/2013 Dept. of ECE 28
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

29. REFERENCES
• WEBSITES
 ieeexplore.ieee.org/
 www.analog.com
 MEMS Accelerometer Based Nonspecific-User Hand Gesture
Recognition , IEEE SENSORS JOURNAL, VOL. 12, NO. 5, MAY 2012.
 S. Zhou, Z. Dong, W. J. Li, and C. P. Kwong, “Hand-written character
recognition using MEMS motion sensing technology,” in Proc.IEEE/ASME
Int. Conf. Advanced Intelligent Mechatronics, 2008
• BOOKS REFERED
25/7/2013 Dept. of ECE 29
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

30. 25/7/2013 Dept. of ECE 30
THANK YOU
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

31. 25/7/2013 Dept. of ECE 31
QUERIES ????.....
MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition