This document discusses techniques for motion analysis and object tracking in video. It covers topics like frame differencing, background subtraction, optical flow, and object tracking methods including point tracking using Kalman filtering, particle filtering, and multiple hypothesis tracking. It also discusses kernel-based tracking and classification techniques like shape-based, motion-based, color-based, and texture-based classification. The goal of object tracking is to estimate an object's position across video frames as it moves.

1. Motion Analysis
and Object Tracking
• The goal of this chapter is to cover motion analysis and
the tracking of objects.
• The following topics are covered in detail in this
chapter:
• Object tracking techniques, including using frame
differencing to learn some information about an object
in motion
• Background and foreground subtraction
• Using optical flow techniques for object feature
tracking
• Building interactive object tracking using the mean
shift and cam shift techniques

2. Introduction to Object Tracking
• Object tracking is the process of estimating
the exact position of an object while the
object is in motion or across consecutive
image frames within a video.
• Figure 1 shows the basic steps for tracking an
object.

3. Applications
• There are many applications of object tracking,
such as security surveillance, augmented reality,
traffic monitoring, self-driving cars, action
recognition, and so on.
• The ability to track an object in a video depends
on multiple factors, such as knowledge about
what the object in context is, what parameters of
the object are being tracked, and what type of
video is showing the object.

4. Challenges of Object Tracking
• the following challenges need to be dealt with:
Object occlusion:
• When the target image is hidden behind
something, it is difficult to both detect and
update when future images come in.
Speed:
• When the motion of an object is fast, the output
video usually is blurred or jittery. Hence, any
sudden changes in the motion of cameras lead to
problems in tracking applications.

5. Challenges of Object Tracking
• Shape:
• Tracking objects that are non rigid (i.e. the
• shape is not constant) will result in failure on object
• detection and thus tracking.
• False positives:
• When there are multiple similar objects,
• it is hard to match which object is targeted in
subsequent images. The tracker may lose the current
object in terms of detection and start tracking a similar
object.

6. Object Detection Techniques for Tracking
• The first step in the process of object tracking
is to identify objects of interest in the video
sequence and to cluster pixels of these
objects.
• Since moving objects are typically the primary
source of information, most methods focus on
the detection of such objects.
• This is also referred to as tracking by detection

7. Object detection techniques
applied for object tracking.

8. Frame Differentiation
• Frame differencing is a technique to measure
the difference between two video frames by
observing the position of one or more objects
in context.
• The pixel definitions are observed for any
changes as this is an indication of changes in
the image. A pixel change indicates a change
in the image.

9. Frame Differentiation
• Frame differentiation is all about determining
the presence of moving objects by calculating
the pixel difference between two consecutive
images in a video.
• Frame differentiation techniques have high
accuracy and relatively lower or moderate
computational time; this method works well
for static backgrounds.

10. Background Subtraction
• Background subtraction is an important
preprocessing technique in vision-based
applications because it helps separate the
background from the foreground in video
streams.

11. Background Subtraction
• An interesting use case for this technique is a
ticket counter where the background is static
but the foreground has visitors coming to the
counter to buy tickets.
• The requirement could be counting the
number of visitors coming to the counter in
the day. In this case, you first need to extract
each person alone.

12. Background Subtraction
• If there is an image or a frame of video that
just has the static background and no moving
visitors, it is a straightforward task because all
you need to do is subtract the new image
from the background to extract the
foreground alone.

13. Optical Flow
• Optical flow denotes the motion of the objects in
an image from one frame to another that is caused
by either the motion of the image or the camera.
• It is represented as a 2D vector field that has each
element representing the movement of the points
from one frame to another. Figure 2 represents the
movement of a ball from one position to another
across five consecutive frames.

14. Optical Flow
Optical flow

15. Optical Flow
• The optical flow method assumes that there is
no change in the pixel intensities of an object
between consecutive frames, and neighboring
pixels also have similar motion.
• Optical flow methods are relatively high in
computational time and moderate on
accuracy.

16. Object Classification
Object classification techniques

17. Shaped-Based Classification
• There are many descriptions of shape information
about motion regions such as the representation
of points, blobs, or boxes to classify a given object.
• Classification is done on every frame for the
object, and the results are stored in a histogram.
• Shape-based classification has a lower
computational time and a relatively lower accuracy
because template matching techniques can be
applied

18. Motion-Based Classification
• Moving objects have a periodic property called
residual flow that can be used for classification.
• Residual flow is used to analyze the rigidity and
periodicity of the moving objects.
• Rigid objects present a little residual flow,
whereas a nonrigid moving object like a human
being has a higher average residual flow and
displays a periodic component.

19. Motion-Based Classification
• Motion-based classification has a high
computational time and relatively lower
accuracy.
• Though it doesn’t require templates, it fails to
identify a static human/nonrigid object.

20. Color-Based Classification
• Color usually is not the most appropriate
feature of an object to use for classification,
but, among all the object features, color is fairly
constant and can also be easily acquired.
• Furthermore, it is one of the features that can
be exploited when needed.
• Color histograms are used to detect and track
vehicles in real time.

21. Color-Based Classification
• A Gaussian distribution model is used to
understand the color distribution in a sequence of
images, which is useful to segment the
background and the object.
• Color-based classification has a higher
computational time and relatively higher accuracy
because template matching techniques can be
applied.

22. Texture-Based Classification
• This techniques uses gradient orientation in
the selected portions of the image.
• This method can result in more accuracy
because it uses overlapping contrast
normalization in a dense grid of uniformly
spaced calls.
• Texture-based classification has a higher
computational time and relatively higher
accuracy than other methods.

23. Object Tracking Methods
• An object is tracked to extract objects,
recognize and track objects, and make
decisions about activities.
• Object tracking, at a high level, can be
classified as point tracking, kernel-based
tracking, and silhouette-based tracking

24. Some Object Tracking Techniques

25. Point Tracking Method
• Point tracking is done using the feature points
of the moving object. There are three
methods for point tracking:
1. Kalman filtering
2. particle filtering
3. and multiple hypothesis.

26. Kalman Filtering
• Kalman filter estimates the state of some quantities by
combining two kinds of information
• It uses a restrictive probability density propagation
algorithm.
• It supports estimation of past, present, and future states
using its efficient recursive estimation techniques.
• The goal of KF is to reduce the noise from measurements
• There are two kinds of equations:
1. time update equations.
2. Measurement update equations.

27. Cont :
• Time update equations provide a future state using
the details of the current state and error covariance
estimations, and measurement update equations
help in the feedback process in the recursive flow.

28. Particle filtering
• Considers a variable at one time and
• generates all the models for that variable.
• This method supports the dynamicity of
variable states and also allows for a new
operation of resampling.
• Particle filtering overcomes the restrictions
that Kalman filters pose because they use
contours, color features, or texture mapping.

29. Multiple hypothesis tracking
• It observes more than one frame for better tracking
results. MHT is an iterative mechanism as well.
• Every iteration starts with an existing track and a
hypothesis that has a set of disconnected tracks.
• For each hypothesis, the future position of the object
in the next frame is predicted.
• Each of these predictions is compared using distance
• measures.
• MHT can track multiple objects and also handle
occlusions.

30. Kernel-Based Tracking Methods
• Kernel-based tracking methods measure a
moving object’s emerging region between
frames.
• The object’s movement can be a parametric
• motion such as a translation, conformal,
affine, and so on.