6. “
An integration of digital information
with live video or the user’s
environment in real time .
7. Why I used augmented reality?
To become a ‘smart human’ from an ordinary human being.
8. Marker And Marker Less Based AR
Maker-based AR is when the
tracked object is black-white
square marker.
Mark less augmented reality is
when the tracked object can be
anything else: picture, human
body, head, eyes, hand or
fingers etc. and on top of that
you add virtual objects.
14. (a) given a captured image
(b) skin color segmentation
(c) distance transform
(d) extract a single connected
component of the hand
1. Hand Segmentation
15. 2. Fingertip Detection –PART(I)
The points with higher curvature
values than a threshold(on any
scale level) are selected as
candidates points.
Computation of candidate points
Directions are indicate by the
cross product of two vectors.
(a) Ellipses are fitted to contour based on
candidates points
16. For initial detection hand has to be
held fingers up.
10 frames are used in initial
fingertip detection
At last fingertips are ordered based
on the index of the thumb
(b) Fingertips are ordered based on detected thumb
2. Fingertip Detection –PART(II)
17. 3. Fingertip Tracking
Matching the newly detected fingertips to the previously
tracked fingertips.
Using matching algorithm that minimizes the displacement
of pairs of fingertips over two frames.
The matching cost is minimized as
18. 4. Pose Estimation from Fingertips
Remove the errors in tracking fingertips
By using Kalman filter the estimate camera pose is smoothed
20. Discussion
Illumination changes causing different hand skin color and for out door scenes.
Implementation will not successfully track two hands in similar area.
When fingertip are not visible, this system determines that it has lost tracking
the fingertips.
21. Credits
Special thanks to all the people who made and
released these awesome resources for free:
Handy AR: Markerless Inspection of Augmented Reality Objects Using Fingertip Tracking
M. Kolsch and M. Turk. Fast 2D hand tracking with flocks ¨of features and multi-cue integration. In
Vision for HumanComputer Interaction, page 158, 2004.
A. A. Argyros and M. I. A. Lourakis. Vision-based interpretation of hand gestures for remote control
of a computer mouse. In Computer Vision in Human-Computer Interaction, pages40–51, 2006.
Letessier, Julien and Berard, Francois. Visual tracking of bare fingers for interactive surfaces. In
Proceedings of the ACM Symposium on User Interface Software and Technology, Interactive surfaces,
pages 119–122, 2004.
Z. Y. Zhang. A flexible new technique for camera calibration. IEEE Trans. Pattern Analysis and
Machine Intelligence, 22(11):1330–1334, Nov. 2000.