Handy Fb (Gesture recognition and facebook manipulation project)
Upcoming SlideShare
Loading in...5

Handy Fb (Gesture recognition and facebook manipulation project)



Used gesture recognition to perform events like " chat, write on wall , like this and poke" on face-book look alike page.

Used gesture recognition to perform events like " chat, write on wall , like this and poke" on face-book look alike page.



Total Views
Views on SlideShare
Embed Views



2 Embeds 7

https://www.linkedin.com 5
http://www.linkedin.com 2



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Handy Fb (Gesture recognition and facebook manipulation project) Handy Fb (Gesture recognition and facebook manipulation project) Presentation Transcript

  • Handy Facebook (Hand gestures to manipulate social networking website)
  • Team Members
    • Jaskaran Uppal (0419)
    • Sandeep Mallela (9769)
    • Darpan Dhamija (0550)
    • Rahul Perhar (4562)
  • Project Objective
    • Identify hand gestures in front of a webcam
    • Navigate the website depending on the gestures recognized
  • Tasks to be performed
    • Making of gestures in front of the camera
    • Gesture detection at a suitable frame rate
    • Capturing the gestures and storing them in a .jpg file
    • System training to recognize the gestures with a low error rate
    • Execution of events upon the successful gesture recognition on the webpage
    • Notification to be sent to the user
  • Gesture Making
    • Usage of a small set of gestures (fingers).
    • Every finger raised will perform some predefined navigation of the webpage
    • System capabilities can be programmed to accommodate other human gestures as well
    • Error in detection can be reduced by training
  • Gesture Detection
    • Gestures are detected at a suitable frame rate.
    • The camera captures the hand gesture and we apply canny edge detection algorithm to store the gestures in the following format
  • System Training
    • System training is done using “Neuroph” an open source Image Recognition tool that takes images as input and produces a neural network.
    • This Neural network can be trained to recognize the gestures
    • This can be used with Java Classes to be integrated in our application, using plug-in provided with the tool
  • Website Navigation
    • The default page shown to the user
    • User makes gesture
    • System recognizes
    • Website navigates
    • Facebook profile loaded
    • Furthermore the user can use other gestures to navigate though additional WebPages
  • Website Navigation Contd.
    • After initial gesture recognition, user is navigated to a personal profile page where he is given additional options
    • The user can make gestures to perform either of the actions
    • Chat
    • Write on wall
    • Like a post
    • Poke a person
  • Implementation Details
    • The application is implemented using the following:
    • OpenCV libraries for gesture recognition code
    • Using Java to capture the image and convert it into a BufferedImage for easy processing
    • Neuroph tool is used to train the system
    • The output from Neuroph is the recognition of Gesture upon which we have actions defined
  • Results
    • Home Page
  • Results
    • Personal Page of a user
  • Results
    • Opening Chat for a user
  • Results
    • Writing on the wall of a user
  • Results
    • “ Like” a user post
  • Results
    • “ Poke” a user
  • Limitations
    • The Limitations to the system includes the following:
    • The error rate in gesture recognition is persistent
    • It is a Lo-Fi prototype of what can be done on a larger scale further improvements can be done
    • Gesture recognition is dependent upon on available light.
  • Future Additions
    • Improvement in Hand gesture recognition. Making the system more refined and gestures easily recognizable
    • We can Integrate this into a number of applications like Google maps to get the address of a particular place.
    • A lot more different gestures can be used and trained in the system
    • We can have a real chat window in the future
  • Credits & References
    • Prof. Suya You, for all the support and knowledge of various User Interface Designs
    • Vijayakumar Gopalakrishnan, TA for giving an initial idea and helping us in realizing the project till the completion
    • Neuroph and related documentation for gesture recognition ( http://neuroph.sourceforge.net/documentation.html )
  • Thank You