This document proposes a system for enabling interactive video through finger gestures. The system tracks objects in video frames using a tracking algorithm and allows users to select tracked objects using swipe gestures over a webcam. It consists of modules for video pre-processing using object tracking, post-processing to focus tracked objects, and detecting swipe gestures using color detection and motion analysis to trigger actions. The system aims to make video marketing more engaging by allowing interactive selection of products from video content.

1. WELCOME

2. VIDEO INTERACTION THROUGH FINGER
TIPS
Coined By:
Nithin Prince John
Roll No : 11
M.Tech CSE
SBCE, Pattoor
Guided By:
Keerthi A S Pillai
Assistant Professor
Dept of CSE
SBCE, Pattoor

3. CONTENTS
 OBJECTIVE
 INTRODUCTION
 PROPOSED SYSTEM
 MODULES OF THE SYSTEM
 RESULT
 CONCLUSION
5/23/2016 Computer Science & Engg 3

4. OBJECTIVE
Leverage existing content, turn them into transactional
content marketing
5/23/2016 4Computer Science & Engg

5. “4 in 10 shoppers visited a store in person or
online as a direct result of watching a video”
5/23/2016 5Computer Science & Engg

6. INTRODUCTION
Today online videos provide a passive user-experience
for users
Interactive video enables users to take actions right
within the video content
Interruption is the core issue
Developed a technique to blend interaction with any
video, new or existing
5/23/2016 6Computer Science & Engg

7. Contd...
Interactivity makes videos more engaging and
immersive
On the back end, items are tracked and hand-
programmed to link out to their relative features
Creates new levels of interactivity, allowing users to
view the product details using swipe motion gesture
Revolutionize, how companies market, advertise, and
sell their products
5/23/2016 7Computer Science & Engg

8. Contd...
In a nutshell, the future of digital marketing
5/23/2016 8Computer Science & Engg

9. PROPOSED SYSTEM
Consist of following modules :
5/23/2016 9Computer Science & Engg
Swipe Motion Gesture Technique
Video Post-processing
Video Pre-processing

10. Contd...
Video Pre –processing Module
Tracking – Learning – Detection (TLD) : A
framework designed for long-term tracking of an
unknown object in a video stream
ROI is defined by a bounding box in a single frame
TLD simultaneously tracks the object, learns its
appearance and detects it whenever it appears in
the video
5/23/2016 10Computer Science & Engg

11. Contd...
5/23/2016 11Computer Science & Engg

12. Contd...
Choosing the video to be pre-processed
5/23/2016 12Computer Science & Engg

13. Contd...
Input : Frames of the chosen video
5/23/2016 13Computer Science & Engg

14. Contd...
Apply TLD to track interesting objects ( dress, shoes,
etc... ) from it
5/23/2016 14Computer Science & Engg

15. Contd...
Output : Provides us the locations of the selected
objects in each frame
5/23/2016 15Computer Science & Engg

16. Contd...
Video Post –processing Module
Input : Output of the TLD algorithm which contains
the tracking information
Use that position information to focus the object of
interest in each frame
Before the viewer, these frames appear in a rate
more than our perception of vision, gives a
sensation of moving of the object
5/23/2016 16Computer Science & Engg

17. Product Selection Technique
Difficulty of selecting moving objects
5/23/2016 17Computer Science & Engg

18. Swipe Motion Gesture
For selecting product, we want to use simple swipe of
the color card in front of the webcam
5/23/2016 18Computer Science & Engg

19. Contd...
Implementation of swipe motion gesture technique
 Take each frame of the video
 Smooth the frame using Gaussian blur
 Convert frame from BGR to HSV color-space
Extract the specified colored object alone using HSV control
panel
Threshold the HSV image for a range of chosen color
5/23/2016 19Computer Science & Engg

20. Contd...
Track the colored object and calculate the co-ordinates
of the position of the center of the blobs as follows
𝑚 𝑝𝑞 = 𝑥=0
𝑀−1
𝑦=0
𝑁−1
𝑥 𝑝
𝑦 𝑞
𝑓 𝑥, 𝑦
𝑥 = 𝑚10 𝑚00
𝑦 = 𝑚01 𝑚00
5/23/2016 20Computer Science & Engg

21. Contd...
Find the distance traveled by the object in first 15 frames
𝑑 = 𝑥2 − 𝑥1
2 + 𝑦2 − 𝑦1
2
Find velocity of the object
𝑣 = 𝑑 𝑡
If velocity > threshold and the gesture made is a swipe
gesture, then the corresponding action is triggered
5/23/2016 21Computer Science & Engg

22. Contd...
Identification of blue color card
5/23/2016 22Computer Science & Engg

23. RESULT
Output of the proposed system
5/23/2016 23Computer Science & Engg

24. CONCLUSION
Uses computer vision technology, TLD algorithm to
track unknown objects throughout the video
Employed swipe motion gesture for real time
interaction experience
Solution to “How can we increase sales and
engagement through traditional video mediums?”
5/23/2016 24Computer Science & Engg

25. REFERENCES
[1] Khalad Hasan, Yang Wang, Wing Kwong and Pourang Irani. Enabling User
Interactions with Video Contents, 2013
[2] Z. Kalal, K. Mikolajczyk, J. Matas, J. Tracking-Learning- Detection. IEEE Transactions
on PAMI, vol.34, no.7, pp.1409-1422, July 2012
[3] Z. Kalal, J. Matas, and K. Mikolajczyk. P-N Learning: Boot-strapping Binary Classifiers
by Structural Constraints. IEEE Confer. on Computer Vision and Pattern Recognition,
2010.
[4] Z. Kalal, K. Mikolajczyk, and J. Matas. Forward-Backward Error: Automatic Detection
of Tracking Failures, Proc. IAPR International Conference on Pattern Recognition, pp.
23-26,2010.
[5] Saikat Basak and Arundhuti Chowdhury . A Vision Interface Framework for Intuitive
Gesture Recognition using Color based Blob Detection. International Journal of Computer
Applications, vol.90, no.15, March 2014
5/23/2016 25Computer Science & Engg

26. ACHIEVEMENTS
Won Best Paper Award for the paper entitled
“Video Interaction Through Finger Tips” in the
IEEE International Conference on Circuit, Power
and Computing Technologies (ICCPCT-2015),
Kanyakumari, Tamilnadu, on March, 2015
Selected to present paper entitled “Turning Existing Video Contents Into
Transactional Content Marketing” at IEEE and SPRINGER Technically Co-
Sponsored Science and Information Conference 2015, LONDON , UK
5/23/2016 26Computer Science & Engg

27. 5/23/2016 27Computer Science & Engg
Thank you for your
attention !
Questions ?