Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.



Published on

Published in: Technology
  • Be the first to comment


  1. 1. “Hand sign detection” David J. RiosDebate and Conversation in English DCM-FIME-UANL november, 2011
  2. 2. What is augmented reality?Add information to the reality we see to knowmore about the things.
  3. 3. Applications
  4. 4. what is Human ComputerInteraction?HCI is the way a person interacts with acomputer, it can be with a mouse, akeyboard, the screen, touchpads, trackballs,and also using gestures.
  5. 5. What is gesture detection?Use a computer to understand the gesturesbeing made by a person, it can be doneusing computer vision, sensors, markers,...
  6. 6. Hand gesture interaction
  7. 7. Sixthsense
  8. 8. Related work● Hirobe et al. [4] have created a mobile interface that tracks the image of the fingers and allows to type on a keyboard in the air and 3D drawings.
  9. 9. ● Finger counter [6] Using a webcam, that counts fingers and interprets specific hand gestures as input to a system.
  10. 10. ● E. Kollorz et. al. [7] clasification of gestures performed of the image projections on the axes x and y. It uses a Photonic-Mixer- Device (PMD) 3-D sensor
  11. 11. Related work camera sensor makers signs following positionKinect infrared Infrared no no yes no lightElliptics no ultrasound no 5 yes noSixthsense webcam no yes 7 yes yesHirobe high- FPGA no no yes yes frame-rate board*Finger count webcam no no 4 no yesKollortz 3-D Photonic- no 12 no yes Mixer- DeviceProposal webcam no no 6 yes no
  12. 12. Introduction● The focus is on the hand gestures detection and augmented reality area .● We want to develop and implement new algorithms for detecting hand gestures for natural interaction with computer systems and augmented reality.● We want to develop an augmented reality prototype to display maps and directions where natural interaction methods based on hand gestures are implemented.
  13. 13. Image resize We reduce the input image to 160 x 120 pixels faster processing.● cv.Resize(source,destination,cv.CV_INTER_ LINEAR)
  14. 14. Skin color filter● Split the image on three channels R,G,B cv.Split(image,R, G, B, None)● Clasify skin and not-skin pixels [5,10] R (x, y) > 95 and G (x, y) > 40 and B (x, y) > 20 and max{R (x, y), G (x, y), B (x, y)} − min{R (x, y), G (x, y), B (x, y)} > 15 and R (x, y) − G (x, y)| > 15 and R (x, y) > G (x, y) and R (x, y) > B (x, y)● Create binary image
  15. 15. Border detection contour = cv.FindContours(binary, storage, cv.CV_RETR_EXTERNAL , cv.CV_CHAIN_APPROX_SIMPLE)Convex hullhull = cv.ConvexHull2 (contour, storage,cv.CV_CLOCKWISE, 1)Defects defects = cv.ConvexityDefects(contour, hull, storage)
  16. 16. Contour Convex hull Defects
  17. 17. Finger estimation● Using the number of defects and the depth of those defects e can estimate the number of fingers. Average depth: = i ∑ d di n i=0... n
  18. 18. Recognized signs
  19. 19. Hand gesture detection procedure
  20. 20. Current workWe create a computer program capable ofdetect hand signs and use the sign detectedto comunicate with other devices like a GPSor a LEGO robot
  21. 21. Applications● GPS control device:
  22. 22. Video:
  23. 23. ● Robot control:
  24. 24. Video:
  25. 25. References[1] R. Azuma, A Survey of Augmented Reality Presence: Teleoperatorsand Virtual Environments, pp. 355–385, August 1997.[2] ARToolKit {}[3] Ebling, Maria R. and Caceres, Ramon, "Gaming and AugmentedReality Come to Location-Based Services", IEEE Pervasive Computing,vol. 9, pp. 5-6, 2010.[4] TERAJIMA , K., T. KOMURO y M. I SHIKAWA, Fast fingertracking system for in-air typing interface , en CHI ’09: Proceedings ofthe 27th international conference extended abstracts on Human factorsin computing systems, ACM, New York, NY, USA, pages. 3739–3744,2009.[5] Mahmoud, Tarek M., "A New Fast Skin Color DetectionTechnique", vol. World Academy of Science, Engineering andTechnology, no. 43, 2008
  26. 26. [6] CRAMPTON , S. C. y M. B ETKE, Counting fingers in real time: Awebcam-based human-computer interface game applications , enProceedings of the Conference on Universal Access in Human-ComputerInteraction, pages. 1357–1361, 2003.[7] Eva Kollorz, Jochen Penne, Joachim Hornegger, and Alexander Barke.2008. Gesture recognition with a Time Flight camera. Int. J. Intell. Syst.Technol. Appl. 5, 3/4 (November 2008), 334-343.[8] Stergiopoulou, E. and Papamarkos, N., "Hand gesture recognition usinga neural network shape fitting technique", Engineering Applications ofArtificial Intelligence, vol. 22, no. 8, pp. 1141--1158, 2009.[9] Vassili, Vladimir Vezhnevets and Sazonov, Vassili and Andreeva, Alla,"A Survey on Pixel-Based Skin Color Detection Techniques", in in Proc.Graphicon-2003, 2003, pp. 85—92.[10] Mahmoud, Tarek M., "A New Fast Skin Color Detection Technique",vol. World Academy of Science, Engineering and Technology, no. 43,2008.