SlideShare a Scribd company logo
A Framework For Dynamic Hand Gesture
Recognition Using Key Frames Extraction
Bhumika Pathak
GLA University
Mathura, India
Presented by:
Neeraj Baghel
M.Tech 1st Year
1
Subhash Chand
Agrawal
GLA Univesity
Mathura, India
Anand Singh Jalal
GLA University,
Mathura, India
Charul Bhatnagar
GLA Univesity
Mathura, India
2015 Fifth National Conference OF IEEE IN Patna, INDIA
OUTLINE
INTRODUCTION
PROPOSED METHODOLOGY
EXPERIMENTAL RESULTS
CONCLUSION
REFERENCES
2
INTRODUCTION
 Gestures are the bodily actions that include movement of hands, face or other parts of
body.
 Hand gestures are the most natural and intuitive way to perform gesture and play an
important role in wide areas like communicating information through sign language,
virtual reality, automated homes etc.,
 In view of such wide and potential applications, researchers are motivated to develop
interfaces for hand gesture recognition.
 The hand gestures can be categorized as static or dynamic.
 While a static gesture is easy to comprehend, dynamic gestures have the challenge of
their spatio-temporal variability.
 Vision based algorithm are effective alternatives of device based methods as they to
bridge the barrier between man and machine without the need of wearing cumbersome
devices.
3
INTRODUCTION CONT…
 Unlike device based techniques, which find hand shape, motion by
the sensors attached to the devices,
 vision based techniques rely on image features like shape, color,
texture, etc.
 Given an image or video sequence, a vision based hand gesture
recognition system aims to develop algorithms which can correctly
identify the meaning of the gestures.
4
PROPOSED METHODOLOGY
5
PROPOSED METHODOLOGY CONT…
 The block diagram of the proposed frame work is presented in Figure 1.
The most important components of this framework are:
A. key frame extraction module.
B. feature extraction module.
C. recognition module.
 For preprocessing, the video is converted from RGB color model to
YCbCr color model.
 YcbCr model is preferred because of the fact that the skin color can be
identified with the chrominance component and it separates the
brightness value from the chrominance values.
6
PROPOSED METHODOLOGY CONT…
 The suitable values for Cb and Cr are: Cb= [77 127] and Cr=[133
173] [8].
 The hand detection is done by eliminating the face as after skin
detection, only hand and face regions are segmented.
 The face is detected as the region of largest connected component
from the first segmented frame and this region is then subtracted
from all the frames of the video thus leaving hand as the object of
interest.
 Figure 2 shows the results of skin segmentation and hand detection
steps.
7
PROPOSED METHODOLOGY CONT…
8
Key frames extraction
 The video sequence of the dynamic signs consists of large number
of frames.
 All of these frames are not needed to be processed essentially in
order to determine the meaning of the performed sign, rather, only
few important frames from the video are sufficient.
 These most important and thus distinguishing frames are known as
key frames.
 In this paper, Author have devised an algorithm for finding the key
frames from the video.
9
Proposed Algorithm for Key Frames Extraction
10
Feature extraction
 This is the most crucial step for gesture recognition as the quality of
results is dependent on the features used.
 The main parameters to be considered are hand shape, hand motion and
hand orientation.
1. Hand Shape
For hand shape, circularity and extent are calculated. Circularity is the
measure of similarity of a shape’s circumference with respect to its
center (1). Extent refers to the ratio of pixels in the region to pixels in
the total bounding box (2).
11
Feature extraction cont…
2) Hand Motion
For hand motion a unique code is generated which is called the motion
detection code.
Eight unique digits (1-8) are assigned to eight different directions in
which the hand moves from one frame to another.
This code uniquely describes the motion trajectory of the hand. If the
number of key frames is ‘n’, the MDC generated is of size ‘n-1’.
3) Hand Orientation
It refers to the direction of change of hand from one frame
to the other.
If (xc,yc) represent the coordinates of the centroid,
then orientation can be calculated as:
12
Classification using SVM
 The Support Vector Machine (SVM) is a machine learning algorithm which
is used for binary classification [9].
 The main concept of SVM is to find a hyperplane that distinguishes the
positive and negative class with the objective to maximize the width of the
hyperplane between the classes.
 For classifying multiple hand gestures, SVM needs to be extended to solve
the classification problems involving multiple classes.
 In the proposed work, Author have utilized one-against-one approach [10].
In this approach, the ijth binary classifier uses the pattern of class i (+1) as
positive examples and the patterns of class j (-1) as negative examples, and
thus forming a feature vector for every class. To find the result, the distance
between the vectors of the test sample is calculated with all the trained
samples.
13
EXPERIMENTAL RESULTS
 Author have taken 22 hand gestures of Indian Sign Language (ISL) for the
performance evaluation.
 In this dataset, various signers perform the different one handed dynamic signs.
 This is done to ensure that multiple users can use the proposed system and it is not
restricted to be used by single user.
 In addition to it, different background conditions having uniform or non-uniform
background are taken into account.
 The dataset is created by copying the signs from the videos available online at
[http://www.indiansignlanguage.org].
14
EXPERIMENTAL RESULTS CONT…
 The system is implemented using MATLAB 2014a.
 Table II gives the result of the recognition accuracy when tested on 22
different hand gestures of ISL.
 Author have evaluated the performance of the entire Sign Language
Recognition System in terms of accuracy.
 The accuracy of proposed system is calculated as follows:
 The experiments show that the overall accuracy of the proposed system is
90.46%.
 Figure shows the comparison number of key frames extracted by Agrawal
et al. in [5] with the proposed algorithm for signs “wrong” and “down”.
15
EXPERIMENTAL RESULTS CONT…
16
EXPERIMENTAL RESULTS CONT…
17
EXPERIMENTAL RESULTS CONT…
18
CONCLUSION
 In this paper, author have proposed a vision based method for recognition of
dynamic hand gestures skin color segmentation
key frames extraction
using multiple features for hand description
multiclass SVM classifier.
 The proposed novel key frame extraction algorithm speeds up the system by
figuring out the most important frames for feature selection.
 The outcomes show that the system is able to recognize varied one handed
dynamic signs of Indian sign language efficiently.
 Author have also compared the performance of the proposed system with the
other approaches and the experiments show the effectiveness of the proposed
algorithm to find key frames while maintaining the recognition rate.
19
REFERENCES
[1] L. Yun, Z. Lifeng, Z. Shujun, “A Hand Gesture Recognition Method Based on Multi-
Feature Fusion and Template Matching,” International Workshop on Information and
Electronics Engineering (IWIEE) pp.1678 –1684, 2012.
[2] T. Bouchrika, M. Zaied, O. Jemai and C. Ben, “Neural solutions to interact with
computers by hand gesture recognition,” Multimedia Tools and Applications archive journal,
Vol. 72, No. 3, pp. 2949-2975, 2014.
[3] U. Lee and J. Tanaka, “Finger identification and hand gesture recognition techniques for
natural user interface,” In Proceedings of the 11th Asia Pacific Conference on Computer
Human Interaction, APCHI,pp. 274-279, 2013.
[4] A. Nandy, J. S. Prasad, S. Mondal, P. Chakraborty and G.C. Nandi, “Recognition of
Isolated Indian Sign Language Gesture in Real Time,”Springer-Verlag Berlin Heidelberg ,
pp. 102–107, 2010.
[5] S. C. Agrawal, A. S. Jalal and C. Bhatnagar, “Redundancy removal for isolated gesture in
Indian sign language and recognition using multi- class support vector machine,” Int. J.
Computational Vision and Robotics, Vol. 4, Nos. 1/2, pp 23-38, 2014.
20
[6] D. Doye and M. Kokare, “Hand Gesture Recognition Using Object Based Key Frame
Selection,”, In Proceedings of International Conference onDigital Image Processing, pp.
288-291, 2009.
[7] M. K. Bhuyan, “FSM-based recognition of dynamic hand gestures via gesture
summarization using key video object planes,” International Journal of Computer and
Communication Engineering, Vol. 1, No.6, pp.248-259. 2012.
[8] D. Chai and K. N. Ngan, “Face segmentation using skin-color map in videophone
applications,” IEEE Trans. Circuits Syst. Video Technol.,vol. 9, pp. 551–564, 1999.
[9] H. Byun and S. W. Lee, “Applications of Support Vector Machines for Pattern Recognition:
A Survey,” in SVM 2002, Vol. LNCS 2388, Springer, pp. 213-236, 2002.
[10] Y. Liu and Y. F. Zheng, “One-against-all multi-class SVM classification using reliability
measures,” Neural Networks, IJCNN '05,Proceedings in IEEE International Joint
Conference, Vol.2, pp. 849-854, 2005.
21
REFERENCES
THANK YOU
22

More Related Content

What's hot

Gesture recognition document
Gesture recognition documentGesture recognition document
Gesture recognition document
Nikhil Jha
 
IRJET- Hand Gesture Recognition System using Convolutional Neural Networks
IRJET- Hand Gesture Recognition System using Convolutional Neural NetworksIRJET- Hand Gesture Recognition System using Convolutional Neural Networks
IRJET- Hand Gesture Recognition System using Convolutional Neural Networks
IRJET Journal
 
Gesture recognition technology
Gesture recognition technology Gesture recognition technology
Gesture recognition technology
Nagamani Gurram
 
Gesture Recognition Technology-Seminar PPT
Gesture Recognition Technology-Seminar PPTGesture Recognition Technology-Seminar PPT
Gesture Recognition Technology-Seminar PPT
Suraj Rai
 
Hand Gesture Recognition using Image Processing
Hand Gesture Recognition using Image ProcessingHand Gesture Recognition using Image Processing
Hand Gesture Recognition using Image Processing
Danda Srinivas Reddy
 
Hand Gesture Recognition Using OpenCV Python
Hand Gesture Recognition Using OpenCV Python Hand Gesture Recognition Using OpenCV Python
Hand Gesture Recognition Using OpenCV Python
Arijit Mukherjee
 
Gesture Recognition Technology
Gesture Recognition TechnologyGesture Recognition Technology
Gesture Recognition Technology
Muhammad Zeeshan
 
Sign language translator ieee power point
Sign language translator ieee power pointSign language translator ieee power point
Sign language translator ieee power point
Madhuri Yellapu
 
GESTURE RECOGNITION TECHNOLOGY
GESTURE RECOGNITION TECHNOLOGYGESTURE RECOGNITION TECHNOLOGY
GESTURE RECOGNITION TECHNOLOGY
jinal thakrar
 
Eye phone report in ieee format
Eye phone report in ieee formatEye phone report in ieee format
Eye phone report in ieee format
sahithi reddy
 
Gesture recognition technology
Gesture recognition technologyGesture recognition technology
Gesture recognition technology
Sahil Abbas
 
hand gestures
hand gestureshand gestures
hand gestures
Rekha Ganesh
 
Hand Gesture Recognition
Hand Gesture RecognitionHand Gesture Recognition
Hand Gesture Recognition
Shounak Katyayan
 
Gesture Recognition
Gesture RecognitionGesture Recognition
Gesture Recognition
Murlidhar Sarda
 
Gesture Recogntion Technology
Gesture Recogntion TechnologyGesture Recogntion Technology
Gesture Recogntion Technology
Mohit Sipani
 
Hand Gesture Recognition system for deaf and dump people
Hand Gesture Recognition system for deaf and dump peopleHand Gesture Recognition system for deaf and dump people
Hand Gesture Recognition system for deaf and dump people
Manu Devappa
 
Gesture recognition
Gesture recognitionGesture recognition
Gesture recognition
PrachiWadekar
 
Gesture Recognition
Gesture RecognitionGesture Recognition
Gesture Recognition
Shounak Katyayan
 
Hand Gesture Recognition System for Human-Computer Interaction with Web-Cam
Hand Gesture Recognition System for Human-Computer Interaction with Web-CamHand Gesture Recognition System for Human-Computer Interaction with Web-Cam
Hand Gesture Recognition System for Human-Computer Interaction with Web-Cam
ijsrd.com
 
Hand gesture recognition
Hand gesture recognitionHand gesture recognition
Hand gesture recognition
bakhti rahman
 

What's hot (20)

Gesture recognition document
Gesture recognition documentGesture recognition document
Gesture recognition document
 
IRJET- Hand Gesture Recognition System using Convolutional Neural Networks
IRJET- Hand Gesture Recognition System using Convolutional Neural NetworksIRJET- Hand Gesture Recognition System using Convolutional Neural Networks
IRJET- Hand Gesture Recognition System using Convolutional Neural Networks
 
Gesture recognition technology
Gesture recognition technology Gesture recognition technology
Gesture recognition technology
 
Gesture Recognition Technology-Seminar PPT
Gesture Recognition Technology-Seminar PPTGesture Recognition Technology-Seminar PPT
Gesture Recognition Technology-Seminar PPT
 
Hand Gesture Recognition using Image Processing
Hand Gesture Recognition using Image ProcessingHand Gesture Recognition using Image Processing
Hand Gesture Recognition using Image Processing
 
Hand Gesture Recognition Using OpenCV Python
Hand Gesture Recognition Using OpenCV Python Hand Gesture Recognition Using OpenCV Python
Hand Gesture Recognition Using OpenCV Python
 
Gesture Recognition Technology
Gesture Recognition TechnologyGesture Recognition Technology
Gesture Recognition Technology
 
Sign language translator ieee power point
Sign language translator ieee power pointSign language translator ieee power point
Sign language translator ieee power point
 
GESTURE RECOGNITION TECHNOLOGY
GESTURE RECOGNITION TECHNOLOGYGESTURE RECOGNITION TECHNOLOGY
GESTURE RECOGNITION TECHNOLOGY
 
Eye phone report in ieee format
Eye phone report in ieee formatEye phone report in ieee format
Eye phone report in ieee format
 
Gesture recognition technology
Gesture recognition technologyGesture recognition technology
Gesture recognition technology
 
hand gestures
hand gestureshand gestures
hand gestures
 
Hand Gesture Recognition
Hand Gesture RecognitionHand Gesture Recognition
Hand Gesture Recognition
 
Gesture Recognition
Gesture RecognitionGesture Recognition
Gesture Recognition
 
Gesture Recogntion Technology
Gesture Recogntion TechnologyGesture Recogntion Technology
Gesture Recogntion Technology
 
Hand Gesture Recognition system for deaf and dump people
Hand Gesture Recognition system for deaf and dump peopleHand Gesture Recognition system for deaf and dump people
Hand Gesture Recognition system for deaf and dump people
 
Gesture recognition
Gesture recognitionGesture recognition
Gesture recognition
 
Gesture Recognition
Gesture RecognitionGesture Recognition
Gesture Recognition
 
Hand Gesture Recognition System for Human-Computer Interaction with Web-Cam
Hand Gesture Recognition System for Human-Computer Interaction with Web-CamHand Gesture Recognition System for Human-Computer Interaction with Web-Cam
Hand Gesture Recognition System for Human-Computer Interaction with Web-Cam
 
Hand gesture recognition
Hand gesture recognitionHand gesture recognition
Hand gesture recognition
 

Similar to A Framework For Dynamic Hand Gesture Recognition Using Key Frames Extraction

TOUCHLESS ECOSYSTEM USING HAND GESTURES
TOUCHLESS ECOSYSTEM USING HAND GESTURESTOUCHLESS ECOSYSTEM USING HAND GESTURES
TOUCHLESS ECOSYSTEM USING HAND GESTURES
IRJET Journal
 
Sign Language Identification based on Hand Gestures
Sign Language Identification based on Hand GesturesSign Language Identification based on Hand Gestures
Sign Language Identification based on Hand Gestures
IRJET Journal
 
Hand Gesture Recognition Using Statistical and Artificial Geometric Methods :...
Hand Gesture Recognition Using Statistical and Artificial Geometric Methods :...Hand Gesture Recognition Using Statistical and Artificial Geometric Methods :...
Hand Gesture Recognition Using Statistical and Artificial Geometric Methods :...
caijjournal
 
IRJET- Vision Based Sign Language by using Matlab
IRJET- Vision Based Sign Language by using MatlabIRJET- Vision Based Sign Language by using Matlab
IRJET- Vision Based Sign Language by using Matlab
IRJET Journal
 
Real Time Sign Language Detection
Real Time Sign Language DetectionReal Time Sign Language Detection
Real Time Sign Language Detection
IRJET Journal
 
Development of Sign Signal Translation System Based on Altera’s FPGA DE2 Board
Development of Sign Signal Translation System Based on Altera’s FPGA DE2 BoardDevelopment of Sign Signal Translation System Based on Altera’s FPGA DE2 Board
Development of Sign Signal Translation System Based on Altera’s FPGA DE2 Board
Waqas Tariq
 
Hand Segmentation for Hand Gesture Recognition
Hand Segmentation for Hand Gesture RecognitionHand Segmentation for Hand Gesture Recognition
Hand Segmentation for Hand Gesture Recognition
AM Publications,India
 
Ay4103315317
Ay4103315317Ay4103315317
Ay4103315317
IJERA Editor
 
IRJET- Gesture Recognition for Indian Sign Language using HOG and SVM
IRJET-  	  Gesture Recognition for Indian Sign Language using HOG and SVMIRJET-  	  Gesture Recognition for Indian Sign Language using HOG and SVM
IRJET- Gesture Recognition for Indian Sign Language using HOG and SVM
IRJET Journal
 
Natural Hand Gestures Recognition System for Intelligent HCI: A Survey
Natural Hand Gestures Recognition System for Intelligent HCI: A SurveyNatural Hand Gestures Recognition System for Intelligent HCI: A Survey
Natural Hand Gestures Recognition System for Intelligent HCI: A Survey
Editor IJCATR
 
Hand Gesture Identification
Hand Gesture IdentificationHand Gesture Identification
Hand Gesture Identification
IRJET Journal
 
IRJET - Paint using Hand Gesture
IRJET - Paint using Hand GestureIRJET - Paint using Hand Gesture
IRJET - Paint using Hand Gesture
IRJET Journal
 
Real Time Sign Language Detection
Real Time Sign Language DetectionReal Time Sign Language Detection
Real Time Sign Language Detection
IRJET Journal
 
Real-Time System of Hand Detection And Gesture Recognition In Cyber Presence ...
Real-Time System of Hand Detection And Gesture Recognition In Cyber Presence ...Real-Time System of Hand Detection And Gesture Recognition In Cyber Presence ...
Real-Time System of Hand Detection And Gesture Recognition In Cyber Presence ...
IJERA Editor
 
Hand Segmentation Techniques to Hand Gesture Recognition for Natural Human Co...
Hand Segmentation Techniques to Hand Gesture Recognition for Natural Human Co...Hand Segmentation Techniques to Hand Gesture Recognition for Natural Human Co...
Hand Segmentation Techniques to Hand Gesture Recognition for Natural Human Co...
Waqas Tariq
 
Sign Language Recognition
Sign Language RecognitionSign Language Recognition
Sign Language Recognition
IRJET Journal
 
Novel Approach to Use HU Moments with Image Processing Techniques for Real Ti...
Novel Approach to Use HU Moments with Image Processing Techniques for Real Ti...Novel Approach to Use HU Moments with Image Processing Techniques for Real Ti...
Novel Approach to Use HU Moments with Image Processing Techniques for Real Ti...
CSCJournals
 
Appearance based static hand gesture alphabet recognition
Appearance based static hand gesture alphabet recognitionAppearance based static hand gesture alphabet recognition
Appearance based static hand gesture alphabet recognition
IAEME Publication
 
Sign Language Recognition using Mediapipe
Sign Language Recognition using MediapipeSign Language Recognition using Mediapipe
Sign Language Recognition using Mediapipe
IRJET Journal
 
Cc4301455457
Cc4301455457Cc4301455457
Cc4301455457
IJERA Editor
 

Similar to A Framework For Dynamic Hand Gesture Recognition Using Key Frames Extraction (20)

TOUCHLESS ECOSYSTEM USING HAND GESTURES
TOUCHLESS ECOSYSTEM USING HAND GESTURESTOUCHLESS ECOSYSTEM USING HAND GESTURES
TOUCHLESS ECOSYSTEM USING HAND GESTURES
 
Sign Language Identification based on Hand Gestures
Sign Language Identification based on Hand GesturesSign Language Identification based on Hand Gestures
Sign Language Identification based on Hand Gestures
 
Hand Gesture Recognition Using Statistical and Artificial Geometric Methods :...
Hand Gesture Recognition Using Statistical and Artificial Geometric Methods :...Hand Gesture Recognition Using Statistical and Artificial Geometric Methods :...
Hand Gesture Recognition Using Statistical and Artificial Geometric Methods :...
 
IRJET- Vision Based Sign Language by using Matlab
IRJET- Vision Based Sign Language by using MatlabIRJET- Vision Based Sign Language by using Matlab
IRJET- Vision Based Sign Language by using Matlab
 
Real Time Sign Language Detection
Real Time Sign Language DetectionReal Time Sign Language Detection
Real Time Sign Language Detection
 
Development of Sign Signal Translation System Based on Altera’s FPGA DE2 Board
Development of Sign Signal Translation System Based on Altera’s FPGA DE2 BoardDevelopment of Sign Signal Translation System Based on Altera’s FPGA DE2 Board
Development of Sign Signal Translation System Based on Altera’s FPGA DE2 Board
 
Hand Segmentation for Hand Gesture Recognition
Hand Segmentation for Hand Gesture RecognitionHand Segmentation for Hand Gesture Recognition
Hand Segmentation for Hand Gesture Recognition
 
Ay4103315317
Ay4103315317Ay4103315317
Ay4103315317
 
IRJET- Gesture Recognition for Indian Sign Language using HOG and SVM
IRJET-  	  Gesture Recognition for Indian Sign Language using HOG and SVMIRJET-  	  Gesture Recognition for Indian Sign Language using HOG and SVM
IRJET- Gesture Recognition for Indian Sign Language using HOG and SVM
 
Natural Hand Gestures Recognition System for Intelligent HCI: A Survey
Natural Hand Gestures Recognition System for Intelligent HCI: A SurveyNatural Hand Gestures Recognition System for Intelligent HCI: A Survey
Natural Hand Gestures Recognition System for Intelligent HCI: A Survey
 
Hand Gesture Identification
Hand Gesture IdentificationHand Gesture Identification
Hand Gesture Identification
 
IRJET - Paint using Hand Gesture
IRJET - Paint using Hand GestureIRJET - Paint using Hand Gesture
IRJET - Paint using Hand Gesture
 
Real Time Sign Language Detection
Real Time Sign Language DetectionReal Time Sign Language Detection
Real Time Sign Language Detection
 
Real-Time System of Hand Detection And Gesture Recognition In Cyber Presence ...
Real-Time System of Hand Detection And Gesture Recognition In Cyber Presence ...Real-Time System of Hand Detection And Gesture Recognition In Cyber Presence ...
Real-Time System of Hand Detection And Gesture Recognition In Cyber Presence ...
 
Hand Segmentation Techniques to Hand Gesture Recognition for Natural Human Co...
Hand Segmentation Techniques to Hand Gesture Recognition for Natural Human Co...Hand Segmentation Techniques to Hand Gesture Recognition for Natural Human Co...
Hand Segmentation Techniques to Hand Gesture Recognition for Natural Human Co...
 
Sign Language Recognition
Sign Language RecognitionSign Language Recognition
Sign Language Recognition
 
Novel Approach to Use HU Moments with Image Processing Techniques for Real Ti...
Novel Approach to Use HU Moments with Image Processing Techniques for Real Ti...Novel Approach to Use HU Moments with Image Processing Techniques for Real Ti...
Novel Approach to Use HU Moments with Image Processing Techniques for Real Ti...
 
Appearance based static hand gesture alphabet recognition
Appearance based static hand gesture alphabet recognitionAppearance based static hand gesture alphabet recognition
Appearance based static hand gesture alphabet recognition
 
Sign Language Recognition using Mediapipe
Sign Language Recognition using MediapipeSign Language Recognition using Mediapipe
Sign Language Recognition using Mediapipe
 
Cc4301455457
Cc4301455457Cc4301455457
Cc4301455457
 

More from NEERAJ BAGHEL

Generating super resolution images using transformers
Generating super resolution images using transformersGenerating super resolution images using transformers
Generating super resolution images using transformers
NEERAJ BAGHEL
 
Mtech Fourth progress presentation
Mtech Fourth progress presentationMtech Fourth progress presentation
Mtech Fourth progress presentation
NEERAJ BAGHEL
 
M.tech Third progress Presentation
M.tech Third progress PresentationM.tech Third progress Presentation
M.tech Third progress Presentation
NEERAJ BAGHEL
 
Latex intro
Latex introLatex intro
Latex intro
NEERAJ BAGHEL
 
Hierarchical structure adaptive
Hierarchical structure adaptiveHierarchical structure adaptive
Hierarchical structure adaptive
NEERAJ BAGHEL
 
Mtech Second progresspresentation ON VIDEO SUMMARIZATION
Mtech Second progresspresentation ON VIDEO SUMMARIZATIONMtech Second progresspresentation ON VIDEO SUMMARIZATION
Mtech Second progresspresentation ON VIDEO SUMMARIZATION
NEERAJ BAGHEL
 
Unsupervised object-level video summarization with online motion auto-encoder
Unsupervised object-level video summarization with online motion auto-encoderUnsupervised object-level video summarization with online motion auto-encoder
Unsupervised object-level video summarization with online motion auto-encoder
NEERAJ BAGHEL
 
Mtech First progress PRESENTATION ON VIDEO SUMMARIZATION
Mtech First progress PRESENTATION ON VIDEO SUMMARIZATIONMtech First progress PRESENTATION ON VIDEO SUMMARIZATION
Mtech First progress PRESENTATION ON VIDEO SUMMARIZATION
NEERAJ BAGHEL
 
Host rank:Exploiting the Hierarchical Structure for Link Analysis
Host rank:Exploiting the Hierarchical Structure for Link AnalysisHost rank:Exploiting the Hierarchical Structure for Link Analysis
Host rank:Exploiting the Hierarchical Structure for Link Analysis
NEERAJ BAGHEL
 
TVSum: Summarizing Web Videos Using Titles
TVSum: Summarizing Web Videos Using TitlesTVSum: Summarizing Web Videos Using Titles
TVSum: Summarizing Web Videos Using Titles
NEERAJ BAGHEL
 
Query focused video summarization
Query focused video summarizationQuery focused video summarization
Query focused video summarization
NEERAJ BAGHEL
 
Traffic behavior of local area network based on
Traffic behavior of local area network based onTraffic behavior of local area network based on
Traffic behavior of local area network based on
NEERAJ BAGHEL
 
Fingerprint recognition
Fingerprint recognitionFingerprint recognition
Fingerprint recognition
NEERAJ BAGHEL
 
Disk scheduling
Disk schedulingDisk scheduling
Disk scheduling
NEERAJ BAGHEL
 
SMOWSER (A VOICE BASED BROWSER)
SMOWSER (A VOICE BASED BROWSER)SMOWSER (A VOICE BASED BROWSER)
SMOWSER (A VOICE BASED BROWSER)
NEERAJ BAGHEL
 
Itvv project ppt
Itvv project pptItvv project ppt
Itvv project ppt
NEERAJ BAGHEL
 

More from NEERAJ BAGHEL (16)

Generating super resolution images using transformers
Generating super resolution images using transformersGenerating super resolution images using transformers
Generating super resolution images using transformers
 
Mtech Fourth progress presentation
Mtech Fourth progress presentationMtech Fourth progress presentation
Mtech Fourth progress presentation
 
M.tech Third progress Presentation
M.tech Third progress PresentationM.tech Third progress Presentation
M.tech Third progress Presentation
 
Latex intro
Latex introLatex intro
Latex intro
 
Hierarchical structure adaptive
Hierarchical structure adaptiveHierarchical structure adaptive
Hierarchical structure adaptive
 
Mtech Second progresspresentation ON VIDEO SUMMARIZATION
Mtech Second progresspresentation ON VIDEO SUMMARIZATIONMtech Second progresspresentation ON VIDEO SUMMARIZATION
Mtech Second progresspresentation ON VIDEO SUMMARIZATION
 
Unsupervised object-level video summarization with online motion auto-encoder
Unsupervised object-level video summarization with online motion auto-encoderUnsupervised object-level video summarization with online motion auto-encoder
Unsupervised object-level video summarization with online motion auto-encoder
 
Mtech First progress PRESENTATION ON VIDEO SUMMARIZATION
Mtech First progress PRESENTATION ON VIDEO SUMMARIZATIONMtech First progress PRESENTATION ON VIDEO SUMMARIZATION
Mtech First progress PRESENTATION ON VIDEO SUMMARIZATION
 
Host rank:Exploiting the Hierarchical Structure for Link Analysis
Host rank:Exploiting the Hierarchical Structure for Link AnalysisHost rank:Exploiting the Hierarchical Structure for Link Analysis
Host rank:Exploiting the Hierarchical Structure for Link Analysis
 
TVSum: Summarizing Web Videos Using Titles
TVSum: Summarizing Web Videos Using TitlesTVSum: Summarizing Web Videos Using Titles
TVSum: Summarizing Web Videos Using Titles
 
Query focused video summarization
Query focused video summarizationQuery focused video summarization
Query focused video summarization
 
Traffic behavior of local area network based on
Traffic behavior of local area network based onTraffic behavior of local area network based on
Traffic behavior of local area network based on
 
Fingerprint recognition
Fingerprint recognitionFingerprint recognition
Fingerprint recognition
 
Disk scheduling
Disk schedulingDisk scheduling
Disk scheduling
 
SMOWSER (A VOICE BASED BROWSER)
SMOWSER (A VOICE BASED BROWSER)SMOWSER (A VOICE BASED BROWSER)
SMOWSER (A VOICE BASED BROWSER)
 
Itvv project ppt
Itvv project pptItvv project ppt
Itvv project ppt
 

Recently uploaded

Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
Brian Pichman
 
Zaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdfZaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdf
AmandaCheung15
 
Garbage In, Garbage Out: Why poor data curation is killing your AI models (an...
Garbage In, Garbage Out: Why poor data curation is killing your AI models (an...Garbage In, Garbage Out: Why poor data curation is killing your AI models (an...
Garbage In, Garbage Out: Why poor data curation is killing your AI models (an...
Zilliz
 
Camunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptxCamunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptx
ZachWylie3
 
Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024
siddu769252
 
Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...
Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...
Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...
OnBoard
 
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
FIDO Alliance
 
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
alexjohnson7307
 
Improving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning ContentImproving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning Content
Enterprise Knowledge
 
Finetuning GenAI For Hacking and Defending
Finetuning GenAI For Hacking and DefendingFinetuning GenAI For Hacking and Defending
Finetuning GenAI For Hacking and Defending
Priyanka Aash
 
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision MakingConnector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
DianaGray10
 
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptxMAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
janagijoythi
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
alexjohnson7307
 
Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3
DianaGray10
 
LeadMagnet IQ Review: Unlock the Secret to Effortless Traffic and Leads.pdf
LeadMagnet IQ Review:  Unlock the Secret to Effortless Traffic and Leads.pdfLeadMagnet IQ Review:  Unlock the Secret to Effortless Traffic and Leads.pdf
LeadMagnet IQ Review: Unlock the Secret to Effortless Traffic and Leads.pdf
SelfMade bd
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Nicolás Lopéz
 
Sonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdfSonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdf
SubhamMandal40
 
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
bellared2
 
Perth MuleSoft Meetup July 2024
Perth MuleSoft Meetup July 2024Perth MuleSoft Meetup July 2024
Perth MuleSoft Meetup July 2024
Michael Price
 
What's New in Teams Calling, Meetings, Devices June 2024
What's New in Teams Calling, Meetings, Devices June 2024What's New in Teams Calling, Meetings, Devices June 2024
What's New in Teams Calling, Meetings, Devices June 2024
Stephanie Beckett
 

Recently uploaded (20)

Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
 
Zaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdfZaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdf
 
Garbage In, Garbage Out: Why poor data curation is killing your AI models (an...
Garbage In, Garbage Out: Why poor data curation is killing your AI models (an...Garbage In, Garbage Out: Why poor data curation is killing your AI models (an...
Garbage In, Garbage Out: Why poor data curation is killing your AI models (an...
 
Camunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptxCamunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptx
 
Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024Generative AI Reasoning Tech Talk - July 2024
Generative AI Reasoning Tech Talk - July 2024
 
Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...
Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...
Mastering Board Best Practices: Essential Skills for Effective Non-profit Lea...
 
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
 
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
 
Improving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning ContentImproving Learning Content Efficiency with Reusable Learning Content
Improving Learning Content Efficiency with Reusable Learning Content
 
Finetuning GenAI For Hacking and Defending
Finetuning GenAI For Hacking and DefendingFinetuning GenAI For Hacking and Defending
Finetuning GenAI For Hacking and Defending
 
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision MakingConnector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
 
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptxMAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
 
Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3
 
LeadMagnet IQ Review: Unlock the Secret to Effortless Traffic and Leads.pdf
LeadMagnet IQ Review:  Unlock the Secret to Effortless Traffic and Leads.pdfLeadMagnet IQ Review:  Unlock the Secret to Effortless Traffic and Leads.pdf
LeadMagnet IQ Review: Unlock the Secret to Effortless Traffic and Leads.pdf
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
 
Sonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdfSonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdf
 
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
 
Perth MuleSoft Meetup July 2024
Perth MuleSoft Meetup July 2024Perth MuleSoft Meetup July 2024
Perth MuleSoft Meetup July 2024
 
What's New in Teams Calling, Meetings, Devices June 2024
What's New in Teams Calling, Meetings, Devices June 2024What's New in Teams Calling, Meetings, Devices June 2024
What's New in Teams Calling, Meetings, Devices June 2024
 

A Framework For Dynamic Hand Gesture Recognition Using Key Frames Extraction

  • 1. A Framework For Dynamic Hand Gesture Recognition Using Key Frames Extraction Bhumika Pathak GLA University Mathura, India Presented by: Neeraj Baghel M.Tech 1st Year 1 Subhash Chand Agrawal GLA Univesity Mathura, India Anand Singh Jalal GLA University, Mathura, India Charul Bhatnagar GLA Univesity Mathura, India 2015 Fifth National Conference OF IEEE IN Patna, INDIA
  • 3. INTRODUCTION  Gestures are the bodily actions that include movement of hands, face or other parts of body.  Hand gestures are the most natural and intuitive way to perform gesture and play an important role in wide areas like communicating information through sign language, virtual reality, automated homes etc.,  In view of such wide and potential applications, researchers are motivated to develop interfaces for hand gesture recognition.  The hand gestures can be categorized as static or dynamic.  While a static gesture is easy to comprehend, dynamic gestures have the challenge of their spatio-temporal variability.  Vision based algorithm are effective alternatives of device based methods as they to bridge the barrier between man and machine without the need of wearing cumbersome devices. 3
  • 4. INTRODUCTION CONT…  Unlike device based techniques, which find hand shape, motion by the sensors attached to the devices,  vision based techniques rely on image features like shape, color, texture, etc.  Given an image or video sequence, a vision based hand gesture recognition system aims to develop algorithms which can correctly identify the meaning of the gestures. 4
  • 6. PROPOSED METHODOLOGY CONT…  The block diagram of the proposed frame work is presented in Figure 1. The most important components of this framework are: A. key frame extraction module. B. feature extraction module. C. recognition module.  For preprocessing, the video is converted from RGB color model to YCbCr color model.  YcbCr model is preferred because of the fact that the skin color can be identified with the chrominance component and it separates the brightness value from the chrominance values. 6
  • 7. PROPOSED METHODOLOGY CONT…  The suitable values for Cb and Cr are: Cb= [77 127] and Cr=[133 173] [8].  The hand detection is done by eliminating the face as after skin detection, only hand and face regions are segmented.  The face is detected as the region of largest connected component from the first segmented frame and this region is then subtracted from all the frames of the video thus leaving hand as the object of interest.  Figure 2 shows the results of skin segmentation and hand detection steps. 7
  • 9. Key frames extraction  The video sequence of the dynamic signs consists of large number of frames.  All of these frames are not needed to be processed essentially in order to determine the meaning of the performed sign, rather, only few important frames from the video are sufficient.  These most important and thus distinguishing frames are known as key frames.  In this paper, Author have devised an algorithm for finding the key frames from the video. 9
  • 10. Proposed Algorithm for Key Frames Extraction 10
  • 11. Feature extraction  This is the most crucial step for gesture recognition as the quality of results is dependent on the features used.  The main parameters to be considered are hand shape, hand motion and hand orientation. 1. Hand Shape For hand shape, circularity and extent are calculated. Circularity is the measure of similarity of a shape’s circumference with respect to its center (1). Extent refers to the ratio of pixels in the region to pixels in the total bounding box (2). 11
  • 12. Feature extraction cont… 2) Hand Motion For hand motion a unique code is generated which is called the motion detection code. Eight unique digits (1-8) are assigned to eight different directions in which the hand moves from one frame to another. This code uniquely describes the motion trajectory of the hand. If the number of key frames is ‘n’, the MDC generated is of size ‘n-1’. 3) Hand Orientation It refers to the direction of change of hand from one frame to the other. If (xc,yc) represent the coordinates of the centroid, then orientation can be calculated as: 12
  • 13. Classification using SVM  The Support Vector Machine (SVM) is a machine learning algorithm which is used for binary classification [9].  The main concept of SVM is to find a hyperplane that distinguishes the positive and negative class with the objective to maximize the width of the hyperplane between the classes.  For classifying multiple hand gestures, SVM needs to be extended to solve the classification problems involving multiple classes.  In the proposed work, Author have utilized one-against-one approach [10]. In this approach, the ijth binary classifier uses the pattern of class i (+1) as positive examples and the patterns of class j (-1) as negative examples, and thus forming a feature vector for every class. To find the result, the distance between the vectors of the test sample is calculated with all the trained samples. 13
  • 14. EXPERIMENTAL RESULTS  Author have taken 22 hand gestures of Indian Sign Language (ISL) for the performance evaluation.  In this dataset, various signers perform the different one handed dynamic signs.  This is done to ensure that multiple users can use the proposed system and it is not restricted to be used by single user.  In addition to it, different background conditions having uniform or non-uniform background are taken into account.  The dataset is created by copying the signs from the videos available online at [http://www.indiansignlanguage.org]. 14
  • 15. EXPERIMENTAL RESULTS CONT…  The system is implemented using MATLAB 2014a.  Table II gives the result of the recognition accuracy when tested on 22 different hand gestures of ISL.  Author have evaluated the performance of the entire Sign Language Recognition System in terms of accuracy.  The accuracy of proposed system is calculated as follows:  The experiments show that the overall accuracy of the proposed system is 90.46%.  Figure shows the comparison number of key frames extracted by Agrawal et al. in [5] with the proposed algorithm for signs “wrong” and “down”. 15
  • 19. CONCLUSION  In this paper, author have proposed a vision based method for recognition of dynamic hand gestures skin color segmentation key frames extraction using multiple features for hand description multiclass SVM classifier.  The proposed novel key frame extraction algorithm speeds up the system by figuring out the most important frames for feature selection.  The outcomes show that the system is able to recognize varied one handed dynamic signs of Indian sign language efficiently.  Author have also compared the performance of the proposed system with the other approaches and the experiments show the effectiveness of the proposed algorithm to find key frames while maintaining the recognition rate. 19
  • 20. REFERENCES [1] L. Yun, Z. Lifeng, Z. Shujun, “A Hand Gesture Recognition Method Based on Multi- Feature Fusion and Template Matching,” International Workshop on Information and Electronics Engineering (IWIEE) pp.1678 –1684, 2012. [2] T. Bouchrika, M. Zaied, O. Jemai and C. Ben, “Neural solutions to interact with computers by hand gesture recognition,” Multimedia Tools and Applications archive journal, Vol. 72, No. 3, pp. 2949-2975, 2014. [3] U. Lee and J. Tanaka, “Finger identification and hand gesture recognition techniques for natural user interface,” In Proceedings of the 11th Asia Pacific Conference on Computer Human Interaction, APCHI,pp. 274-279, 2013. [4] A. Nandy, J. S. Prasad, S. Mondal, P. Chakraborty and G.C. Nandi, “Recognition of Isolated Indian Sign Language Gesture in Real Time,”Springer-Verlag Berlin Heidelberg , pp. 102–107, 2010. [5] S. C. Agrawal, A. S. Jalal and C. Bhatnagar, “Redundancy removal for isolated gesture in Indian sign language and recognition using multi- class support vector machine,” Int. J. Computational Vision and Robotics, Vol. 4, Nos. 1/2, pp 23-38, 2014. 20
  • 21. [6] D. Doye and M. Kokare, “Hand Gesture Recognition Using Object Based Key Frame Selection,”, In Proceedings of International Conference onDigital Image Processing, pp. 288-291, 2009. [7] M. K. Bhuyan, “FSM-based recognition of dynamic hand gestures via gesture summarization using key video object planes,” International Journal of Computer and Communication Engineering, Vol. 1, No.6, pp.248-259. 2012. [8] D. Chai and K. N. Ngan, “Face segmentation using skin-color map in videophone applications,” IEEE Trans. Circuits Syst. Video Technol.,vol. 9, pp. 551–564, 1999. [9] H. Byun and S. W. Lee, “Applications of Support Vector Machines for Pattern Recognition: A Survey,” in SVM 2002, Vol. LNCS 2388, Springer, pp. 213-236, 2002. [10] Y. Liu and Y. F. Zheng, “One-against-all multi-class SVM classification using reliability measures,” Neural Networks, IJCNN '05,Proceedings in IEEE International Joint Conference, Vol.2, pp. 849-854, 2005. 21 REFERENCES