“Hand sign detection”          David J. RiosDebate and Conversation in English        DCM-FIME-UANL         november, 2011
What is augmented reality?Add information to the reality we see to knowmore about the things.
Applications
what is Human ComputerInteraction?HCI is the way a person interacts with acomputer, it can be with a mouse, akeyboard, the...
What is gesture detection?Use a computer to understand the gesturesbeing made by a person, it can be doneusing computer vi...
Hand gesture interaction
Sixthsense
Related work●   Hirobe et al. [4] have created a mobile    interface that tracks the image of the fingers    and allows to...
●   Finger counter [6] Using a webcam, that    counts fingers and interprets specific hand    gestures as input to a system.
●   E. Kollorz et. al. [7] clasification of gestures    performed of the image projections on the    axes x and y. It uses...
Related work                 camera      sensor      makers   signs   following   positionKinect           infrared    Inf...
Introduction●   The focus is on the hand gestures    detection and augmented reality area .●   We want to develop and impl...
Image resize    We reduce the input image to 160 x 120    pixels faster processing.●   cv.Resize(source,destination,cv.CV_...
Skin color filter●   Split the image on three channels R,G,B    cv.Split(image,R, G, B, None)●   Clasify skin and not-skin...
Border detection   contour = cv.FindContours(binary, storage,  cv.CV_RETR_EXTERNAL ,  cv.CV_CHAIN_APPROX_SIMPLE)Convex hul...
Contour   Convex          hull          Defects
Finger estimation●   Using the number of defects and the depth    of those defects e can estimate the number    of fingers...
Recognized signs
Hand gesture detection procedure
Current workWe create a computer program capable ofdetect hand signs and use the sign detectedto comunicate with other dev...
Applications●   GPS control device:
Video:         http://www.youtube.com/watch?v=lt_yv4lRMxw
●   Robot control:
Video:         http://www.youtube.com/watch?v=dTS1ejXeOk4
References[1] R. Azuma, A Survey of Augmented Reality Presence: Teleoperatorsand Virtual Environments, pp. 355–385, August...
[6] CRAMPTON , S. C. y M. B ETKE, Counting fingers in real time: Awebcam-based human-computer interface game applications ...
Session6
Session6
Session6
Session6
Session6
Session6
Upcoming SlideShare
Loading in …5
×

Session6

292
-1

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
292
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Session6

  1. 1. “Hand sign detection” David J. RiosDebate and Conversation in English DCM-FIME-UANL november, 2011
  2. 2. What is augmented reality?Add information to the reality we see to knowmore about the things.
  3. 3. Applications
  4. 4. what is Human ComputerInteraction?HCI is the way a person interacts with acomputer, it can be with a mouse, akeyboard, the screen, touchpads, trackballs,and also using gestures.
  5. 5. What is gesture detection?Use a computer to understand the gesturesbeing made by a person, it can be doneusing computer vision, sensors, markers,...
  6. 6. Hand gesture interaction
  7. 7. Sixthsense
  8. 8. Related work● Hirobe et al. [4] have created a mobile interface that tracks the image of the fingers and allows to type on a keyboard in the air and 3D drawings.
  9. 9. ● Finger counter [6] Using a webcam, that counts fingers and interprets specific hand gestures as input to a system.
  10. 10. ● E. Kollorz et. al. [7] clasification of gestures performed of the image projections on the axes x and y. It uses a Photonic-Mixer- Device (PMD) 3-D sensor
  11. 11. Related work camera sensor makers signs following positionKinect infrared Infrared no no yes no lightElliptics no ultrasound no 5 yes noSixthsense webcam no yes 7 yes yesHirobe high- FPGA no no yes yes frame-rate board*Finger count webcam no no 4 no yesKollortz 3-D Photonic- no 12 no yes Mixer- DeviceProposal webcam no no 6 yes no
  12. 12. Introduction● The focus is on the hand gestures detection and augmented reality area .● We want to develop and implement new algorithms for detecting hand gestures for natural interaction with computer systems and augmented reality.● We want to develop an augmented reality prototype to display maps and directions where natural interaction methods based on hand gestures are implemented.
  13. 13. Image resize We reduce the input image to 160 x 120 pixels faster processing.● cv.Resize(source,destination,cv.CV_INTER_ LINEAR)
  14. 14. Skin color filter● Split the image on three channels R,G,B cv.Split(image,R, G, B, None)● Clasify skin and not-skin pixels [5,10] R (x, y) > 95 and G (x, y) > 40 and B (x, y) > 20 and max{R (x, y), G (x, y), B (x, y)} − min{R (x, y), G (x, y), B (x, y)} > 15 and R (x, y) − G (x, y)| > 15 and R (x, y) > G (x, y) and R (x, y) > B (x, y)● Create binary image
  15. 15. Border detection contour = cv.FindContours(binary, storage, cv.CV_RETR_EXTERNAL , cv.CV_CHAIN_APPROX_SIMPLE)Convex hullhull = cv.ConvexHull2 (contour, storage,cv.CV_CLOCKWISE, 1)Defects defects = cv.ConvexityDefects(contour, hull, storage)
  16. 16. Contour Convex hull Defects
  17. 17. Finger estimation● Using the number of defects and the depth of those defects e can estimate the number of fingers. Average depth: = i ∑ d di n i=0... n
  18. 18. Recognized signs
  19. 19. Hand gesture detection procedure
  20. 20. Current workWe create a computer program capable ofdetect hand signs and use the sign detectedto comunicate with other devices like a GPSor a LEGO robot
  21. 21. Applications● GPS control device:
  22. 22. Video: http://www.youtube.com/watch?v=lt_yv4lRMxw
  23. 23. ● Robot control:
  24. 24. Video: http://www.youtube.com/watch?v=dTS1ejXeOk4
  25. 25. References[1] R. Azuma, A Survey of Augmented Reality Presence: Teleoperatorsand Virtual Environments, pp. 355–385, August 1997.[2] ARToolKit {http://www.hitl.washington.edu/artoolkit/}[3] Ebling, Maria R. and Caceres, Ramon, "Gaming and AugmentedReality Come to Location-Based Services", IEEE Pervasive Computing,vol. 9, pp. 5-6, 2010.[4] TERAJIMA , K., T. KOMURO y M. I SHIKAWA, Fast fingertracking system for in-air typing interface , en CHI ’09: Proceedings ofthe 27th international conference extended abstracts on Human factorsin computing systems, ACM, New York, NY, USA, pages. 3739–3744,2009.[5] Mahmoud, Tarek M., "A New Fast Skin Color DetectionTechnique", vol. World Academy of Science, Engineering andTechnology, no. 43, 2008
  26. 26. [6] CRAMPTON , S. C. y M. B ETKE, Counting fingers in real time: Awebcam-based human-computer interface game applications , enProceedings of the Conference on Universal Access in Human-ComputerInteraction, pages. 1357–1361, 2003.[7] Eva Kollorz, Jochen Penne, Joachim Hornegger, and Alexander Barke.2008. Gesture recognition with a Time Flight camera. Int. J. Intell. Syst.Technol. Appl. 5, 3/4 (November 2008), 334-343.[8] Stergiopoulou, E. and Papamarkos, N., "Hand gesture recognition usinga neural network shape fitting technique", Engineering Applications ofArtificial Intelligence, vol. 22, no. 8, pp. 1141--1158, 2009.[9] Vassili, Vladimir Vezhnevets and Sazonov, Vassili and Andreeva, Alla,"A Survey on Pixel-Based Skin Color Detection Techniques", in in Proc.Graphicon-2003, 2003, pp. 85—92.[10] Mahmoud, Tarek M., "A New Fast Skin Color Detection Technique",vol. World Academy of Science, Engineering and Technology, no. 43,2008.

×