This document provides an overview of various object detection and tracking methods. It discusses Scale-Invariant Feature Transform (SIFT) for object detection, which extracts keypoints that are invariant to scale, orientation and illumination changes. It also covers You Only Look Once (YOLO) for real-time object detection using a single neural network, and Histogram of Oriented Gradients (HOG) for human detection. For object tracking, it discusses point tracking using deterministic and statistical methods, kernel tracking using template matching, and silhouette tracking for complex object shapes. Applications mentioned include video surveillance, autonomous vehicles, and human-computer interaction.

1. A Lecture On
VARIOUS OBJECT DETECTION AND TRACKING
METHODS
Presented by ,
Sujeesh kumar j
M2 CSIP

2. OUTLINE
• Object Detection and Methods
• Object Tracking and Methods

3. OBJECT DETECTION
• It detecting instances of semantic objects of a certain class (such as humans, buildings, or
vehicles) in digital videos and images.
• It has proved to be a prominent module for numerous important applications like video
surveillance, autonomous driving, face detection, etc.
• Feature detectors such as Scale Invariant Feature Transform and Speeded Up Robust Feature are
good methods which yield high quality features but it requires high computational time when it
works in real time.
• Based on the information support vector machine and back-propagation neural network training
are performed for the efficient recognition of objects.

4. OBJECT DETECTION METHODS
1. SCALE-INAVARIENT FEATURE TRANSFORM(SIFT)
2. YOU LOOK ONLY ONCE (YOLO)
3. HISTOGRAM OF GRADIENT(HOG)

5. 1. SCALE-INAVARIENT FEATURE TRANSFORM(SIFT)
• The SIFT method can robustly identify objects even among clutter and under partial occlusion
because the SIFT feature descriptor is invariant to scale, orientation, and affine distortion.
• It is used in application of scale of an image.
• It also used to detect corners, circles, blobs etc.
• Procedure here is a multi step process

6. SIFT PROCEDURES
1. SCALE SPACE EXTREMA DETECTION.
2. KEYPOINT LOCALIZATION
3. ORIENTATION ASSIGNMENT
4. KEYPOINT DESCRIPTOR
5. KEYPOINT MATCHING

7. 1. SCALE SPACE EXTREMA DETECTION.
• Once the image is blurred using Difference of Gaussian blurring, the pixel is compared with 8
neighbours.
• Also compared with nine pixels in next and previous scale, if it is a local extrema ,it is a
potential keypoint(best keypoint in scale).

8. 2. KEYPOINT LOCALIZATION
• If the intensity at the extrema is less than the threshold value, they are rejected.
• Edges are removed using eigen values and their ratios.
• Removal of low contrast keypoint and edge point. Strong interest point.

9. 3. ORIENTATION ASSIGNMENT
• Orientation is assigned to each keypoint.
• Orientation histogram created. Highest peak in the histogram or any peak above 80% is
considered.
• Created keypoint with same location and scale but different directions.

10. 4. KEYPOINT DESCRIPTOR AND 5. KEYPOINT MATCHING
• Descriptors are vectors of size (no: of keypoints *128) achieved from orientation histogram.
• 128 = ?
• 16*16 neighbourhood and 16 sub-blocks and each sub-block here 4*4 = 8bins orientation.
• Keypoint matching between two images is done by identifying the nearest neighbours. Further
ratio analysis between closest and second closest is done.
a

11. 2. YOU LOOK ONLY ONCE (YOLO)
• YOLO is a new and a novel approach to object detection.
• In earlier work on object detection repurposes classifiers to perform detection.
• YOLO frames object detection as a regression problem to spatially separated bounding boxes
and associated class probabilities.
• A single neural network predicts bounding boxes and class probabilities directly from full
images in one evaluation.

12. • The whole detection pipeline is a single network, it can be optimized end-to-end directly on
detection performance.
• It is used in real time object detection.
• The network divides the image into regions and predict the bounding boxes and probabilities of
each region.
• Bounding boxes are weighted with predicted probabilities.

13. 3. HISTOGRAM OF GRADIENT(HOG)
• It is used for human detection.
A. HoG Feature Extraction
• Compute centered and horizontal gradient with no smoothing.
• Compute gradient orientation and magnitude.
• For color image ,pick the color channel with highest magnitude for each pixel.
• Computing gradient :
• Centered function f’(x)=lim(h->0)(f(x+h)-f(x-h))÷2h
• Filter masks in both x and y direction
• Calculated magnitude and orientation.

14. Blocks and Cells
• 16*16 blocks of 50% overlap.
• 7*15 = 120 blocks in total.
• Each block should consist of 2*2 cells and
ie. With size 8*8

15. Trilinear Interpolation
• Each blocks consist of 2*2 cells with size 8*8.
• Quantize the gradient orientation into 9bins (0-180).
• Interpolate votes linearly between neighbouring bin centre's.
• Eg: θ = 85 degree
• Distance to the bin center bin70 and bin90 respectively.
• They have degrees 15 and 5.
• Hence ratio is 5/20 = 1/4
15/20 = 3/4
• Vote can be weighted with gaussian to down weight the pixels near the edges of the block.

16. APPLICATIONS
• Traffic monitoring
• Human action recognition
• Used in application of computer vision such as digital forensics etc.

17. OBJECT TRACKING
• Estimating the trajectory of an object over time by locating its position in every frame.
• Estimating the trajectory of an object in the image plane as it moves around a scene.
• Important task within the field of computervision
• There are three key steps in video analysis:
 Detection of interesting moving objects
 Tracking of objects from frame to frame
 Objects tracks recognition

18. OBJECT TRACKING
• Difficulties in tracking objects can arise due to
 Abrupt object motion
 Changing appearance patterns of both the object and thescene.

19. APPLICATION
• Vehicle Navigation, that is, video-based path planning and obstacle avoidance capabilities.
• Motion-Based Recognition ,human identification based on gait, automatic object detection.
• Automated Surveillance, that is, monitoring to detect suspicious activities.
• Human-Computer Interaction, that is, gesture recognition.
• Traffic monitoring, that is, real-time gathering of traffic statistics to direct traffic flow.

20. OBJECT DETECTION IN TRACKING
1. Background Subtraction
• It identifies the object from the video frame.
• It is technique for separating out foreground element from the background and is done by
generating a foreground mask.
• Used for detecting dynamically moving objects from static cameras.
• Several techniques used in background subtraction .Eg: Running average.
• Dst(x,y) = (1-alpha)*dst(x,y) + alpha*src(x,y)
src=source image. dst = destination image. alpha = weight of input image.

21. • Fundaments logic of moving object detection from the difference between the current and a
reference frame.
• Principles :
• It should segment objects of interest when they first appear in scene.
• The background model adapt for both sudden and gradual changes in background.
• Simple Approach:
• Estimate the background for time ‘t’.
• Subtract the background from input frame.
• Apply threshold to the absolute difference to get the foreground mask.

22. 2. Object representation
• In a tracking, object is anything that is of interest for further analysis.
• For instance, boats on the sea, fish inside an aquarium, vehicles on a road, planes in the air,
people walking on a road are a set of objects that may be important to track in a specific
domain.
• Objects can be represented by shapes and appearances.
 Points
 Primitive geometric shapes
 Object silhouette and
• contour
 Articulated shape models
 Skeletal models

24. 3. Segmentation
• Image segmentation algorithms is to partition the image into perceptually similar regions.
• Every segmentation algorithm addresses two problems the criteria for a good partition and
the method for achieving efficient partitioning.
• Mean Shift Clustering For the image segmentation problem, the mean-shift approach to find
clusters in the joint spatial+color space, [l , u, v,x, y], where [l , u, v] represents the color and [x,
y] represents the spatial location.

25. • Using Graph Cut
• A cut in a graph is a set of edges whose removal disconnects the graph.
• Image segmentation can also be formulated as a graph partitioning problem, where the vertices
(pixels), V ={u, v, . . .}, of a graph (image), G, are partitioned into N disjoint sub graphs
(regions), Ai , N
i = 1 Ai = V, Ai ∩ Aj = ∅, i = j.
• Limitation of minimum cut is its bias toward over segmenting the image
• Shi and Malik [2000] propose the normalized cut to overcome the over segmentation
problem.

26. OBJECT TRACKING METHODS

27. 1. POINT TRACKING
A. Deterministic method
• Deterministic methods define a cost function which is made up of constraints like maximum
velocity, common motion and rigidity.
• A greedy algorithm can be used for this which iteratively optimizes point correspondences.
• This cost function must then be minimized for tracking
• The algorithm is modified to preserve a lot of motion information so that point measurements
are not missed.
• Proximity assumes location of object would not change notably from one frame to other.
• Small velocity change (smooth motion) assumes direction and speed of object does not change
drastically.

28. B. Statistical methods
• Statistical methods models uncertainties to handle noise in an image.
• A well-known method for statistical point tracking is multiple hypothesis tracking(MHT).
• A set of hypotheses are designed for an object and predictions are made for each hypothesis for
the object's position.
• The hypothesis with the highest prediction is the most likely and is chosen for tracking.
• For tracking single objects are the Kalman filter and Particle filters. The Kalman filter is
limited to a linear system and uses prediction and correction to estimate an object's motion

29. 2. KERNAL TRACKING
• Represents object as a geometric shape, called a kernel, and estimates motion of this kernel in
consecutive frames.
• KT commonly used to track a single object. Uses brute force to search an image for a region
that matches the template in the previous image.
• The brute force searching results in this method computationally expensive, but this can be
overcome by optimizations to the method, such as limiting the search to a certain region.
• A limitation of kernel tracking is that parts of the background may appear inside the kernel,
but this can be overcome by making the kernel inside the object, instead of around it.

30. 3. SILHOUETTE TRACKING
• Objects have complex shapes, for example, hands, head, and shoulders cannot be well
described by simple geometric shapes. Silhouette based methods provide an accurate shape
description for these objects.
• This model can be in the form of a color histogram, object edges or the object contour. We
divide silhouette trackers into two categories shape matching and contour tracking.
 Shape Matching can be performed similar to tracking based on template matching where an
object silhouette and its associated model is searched in the current frame.
 Contour Tracking
• methods, in contrast to shape matching methods. iteratively evolve an initial contour in the
previous frame to its new position in the current frame.
• Silhouette tracking is employed when tracking the complete region.

31. THANK YOU