This presentation introduces augmented reality (AR) and discusses a markerless AR inspection method using fingertip tracking. AR integrates digital information with the real world in real time. The presentation covers marker-based and markerless AR, and presents a flowchart for a markerless inspection method using hand segmentation, fingertip detection and tracking, and camera pose estimation to overlay virtual objects on a user's hand from different viewing angles. Challenges discussed include illumination changes affecting skin color detection and the inability to track multiple hands. The presentation concludes by thanking sources that provided resources to develop the fingertip tracking AR inspection method.

2. Hello!
I am Tharuka Dilani
I am here for my image processing
presentation.

4. Agenda
Augmented Reality ?
Marker and Markerless based AR
Introduction to the inspection
Methodalogies
Application
Discussion

5. WHAT IS AUGMENTED
REALITY (AR) ?

6. “
An integration of digital information
with live video or the user’s
environment in real time .

7. Why I used augmented reality?
To become a ‘smart human’ from an ordinary human being.

8. Marker And Marker Less Based AR
Maker-based AR is when the
tracked object is black-white
square marker.
Mark less augmented reality is
when the tracked object can be
anything else: picture, human
body, head, eyes, hand or
fingers etc. and on top of that
you add virtual objects.

9. Maker-based AR

10. Marker less AR

11. Markerless Inspection Of
Augmented Reality Objects Using
Fingertip Tracking

12. Inspecting a vertual bunny on top of the user’s hand from
different viewing angles

13. Flowchart
Captured Image
Hand Segmentation
Fingertip Detection
Fingertip Tracking
Camera Pose Estimation

14. (a) given a captured image
(b) skin color segmentation
(c) distance transform
(d) extract a single connected
component of the hand
1. Hand Segmentation

15. 2. Fingertip Detection –PART(I)
 The points with higher curvature
values than a threshold(on any
scale level) are selected as
candidates points.
 Computation of candidate points
 Directions are indicate by the
cross product of two vectors.
(a) Ellipses are fitted to contour based on
candidates points

16.  For initial detection hand has to be
held fingers up.
 10 frames are used in initial
fingertip detection
 At last fingertips are ordered based
on the index of the thumb
(b) Fingertips are ordered based on detected thumb
2. Fingertip Detection –PART(II)

17. 3. Fingertip Tracking
 Matching the newly detected fingertips to the previously
tracked fingertips.
 Using matching algorithm that minimizes the displacement
of pairs of fingertips over two frames.
 The matching cost is minimized as

18. 4. Pose Estimation from Fingertips
 Remove the errors in tracking fingertips
 By using Kalman filter the estimate camera pose is smoothed

19. summary

20. Discussion
 Illumination changes causing different hand skin color and for out door scenes.
 Implementation will not successfully track two hands in similar area.
 When fingertip are not visible, this system determines that it has lost tracking
the fingertips.

21. Credits
Special thanks to all the people who made and
released these awesome resources for free:
 Handy AR: Markerless Inspection of Augmented Reality Objects Using Fingertip Tracking
 M. Kolsch and M. Turk. Fast 2D hand tracking with flocks ¨of features and multi-cue integration. In
Vision for HumanComputer Interaction, page 158, 2004.
 A. A. Argyros and M. I. A. Lourakis. Vision-based interpretation of hand gestures for remote control
of a computer mouse. In Computer Vision in Human-Computer Interaction, pages40–51, 2006.
 Letessier, Julien and Berard, Francois. Visual tracking of bare fingers for interactive surfaces. In
Proceedings of the ACM Symposium on User Interface Software and Technology, Interactive surfaces,
pages 119–122, 2004.
 Z. Y. Zhang. A flexible new technique for camera calibration. IEEE Trans. Pattern Analysis and
Machine Intelligence, 22(11):1330–1334, Nov. 2000.

22. Thanks!
Any questions?
You can find me at
tharukasiriwardena27@gmail.com