3. INTRODUCTION
Object tracking has been widely studied,
It can be a individual application
A very important pre-process for further analysis
Behavior analysis
Event detection
Object recognition
Surveillance applications
And more …
3 CCU CSIE
4. INTRODUCTION
Major issues on object tracking
Occlusion
Target characteristic changes
Environment changes
Features selected greatly affect performance
Every feature has its own limits
Color values
Edges
Created histograms
Hybrid Features
4 CCU CSIE
5. MOTIVATION
CCU CSIE5
For Video Inpainting needs
No still background environment
Target is usually human (non-rigid object)
More detailed segmentation result
6. MOTIVATION – TARGET REPRESENTATION
Target presents in form of simple geometric
shapes, such as ellipse and rectangle.
Easy to maintain
Not suitable for non-rigid targets.
Target should be presented in more detailed way,
not in those simple geometric shapes.
6 CCU CSIE
8. RELATED WORK – ADABOOST-BASED TRACKING
8
In Ensemble Tracking[4]
Use Adaboost-based method to do online tracking
Feature space: f = [h1 … h8 r g b]T
Classifiers: W = [w1 … w11 ]T
How to build the feature pool?
Target presents in rectangle
CCU CSIE
Shai Avidan, “Ensemble Tracking,” IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol. 29, no. 2, February 2007.
9. RELATED WORK – IMAGE SEGMENTATION TOOL
Lazy snapping[5] provides accurate segmentations
for single frame.
Coarse-to-fine strategy
First, segment automatically
Second, edit object boundary manually
However, it is not affordable to do human
interactions frame by frame.
9 CCU CSIE
Yin Li, Jian Sun, Chi-Keung Tang, and Heung-Yeung Shum, “Lazy Snapping,” ACM
Transactions on Graphics, vol. 23, no. 3, August 2004.
11. PROPOSED METHOD
We propose a tracking mechanism, which
Consists of two different trackers
Targets are not presented in simple geometric shapes
No background information is used
Human interaction available
Initialization target(s) in the first frame
Update target information
11 CCU CSIE
14. GLOBAL-VIEWED TRACKER
Pre-process
Down-sample to make
incoming seed features
analysis practicable.
Major tracker
Adaboost on seed features
Post-process
Morphological operations
Global-Viewed Tracker
Pixel-Based
Seed Features
Adaboost
Morphological
Post-process
Down-Sample
Pixel-
based
15 CCU CSIE
15. SEED FEATURES
The seed features[1] is proposed by Collins et al.
It is a mechanism to generate many extended features
from existing features.
It has potential to provide much detailed segmentation
of target.
Robert T. Collins, Yanxi Liu, and Marius Leordeanu, “Online Selection of Discriminative
Tracking Features,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.
27, no. 10, October 2005.
16 CCU CSIE
18. TUNED FEATURES [1]
For each seed feature,
We create the tuned feature in form of
bins
foreground
p(x)
background
q(x)
max ,
log
max ,
p i
L i
q i
19 CCU CSIE
21. THE MOST DISCRIMINATED?
In [1], a discriminability function is defined to find the
most discriminative 5 for further tracking.
To define the discrimination is hard and unreasonable.
Instead, combine them together
23 CCU CSIE
22. ADABOOST
We use Adaboost to combine all seed features
together; trying to generate a more robust one.
Every seed feature is considered as a weak
classifier.
Through Adaboost, we generate a strong
classifier to separate foregrounds and
backgrounds.
24 CCU CSIE
23. 3 THRESHOLDS WEAK CLASSIFIERS
To make our solution space as big as possible,
we use 3 thresholds on every seed features, to
provide more solutions.
Totally 147 weak classifiers are generated.
25 CCU CSIE
bins
27. GLOBAL-VIEWED TRACKER RESULTS
Some foreground colors
are sacrificed.
The damage will be more
if we keep them surviving.
Some background colors
survived.
The damage will be more
if we erase them.
29 CCU CSIE
28. LOCAL-VIEWED TRACKER
Goal: to provide compensation to global-viewed
tracker
To remove false-positive regions
To recover false-negative regions
To cooperate with spatial information
Regionalize to achieve
30 CCU CSIE
32. REGIONALIZATION
Use k-means clustering to achieve regionalization
Each pixel and means are represented as:
The kernel distance function is defined as:
where
2 2 2
1 2 1 2 1 2 1 2
2 2
1 2 1 2 1 2
,
,
T
color location
color
location
x y H S V
D D D
D H H S S V V
D x x y y
p
p p
p p
34 CCU CSIE
33. REGIONALIZATION USING K-MEANS CLUSTERING
Two advantage
Time saving from impossible
candidates
Strengthening physical constrains
Color variety measure
color locationD D D
22 21
var
3 i j
j i i i
j
H H S S V V
n
r
35 CCU CSIE
34. REGIONALIZATION ISSUE
Emphasize on location distance?
Regions are compact so that easy for
human interaction.
Regions will be forced to sacrifice
detailed color information, such as
edges.
color locationD D D
36 CCU CSIE
35. DYNAMIC KERNEL DISTANCE FUNCTION
To enforce color distance part will be better.
K-means clustering itself is not robust due to
unpredictable-changed regions.
Location Distance
Color Distance
Region Compactness
Color Regularity
color locationD D D
37 CCU CSIE
38. TRACKING USING K-MEANS CLUSTERING
The tracking problem can be considered as a
mapping problem.
We use k-means clustering to achieve this
mapping.
40 CCU CSIE
t t+1
39. BIDIRECTIONAL CLUSTERING
Backward
The region in
current frame
comes from one
in previous frame
Forward
The region in
previous frame
goes to one in
current frame
t – 1
t
41 CCU CSIE
42. ADABOOST ON REGION-BASED SEED FEATURES
Use the adaboost on seed features again
To strengthen the decision we make from
previous bidirectional clustering.
44 CCU CSIE
43. REGION RELATION
Inspired by lazy snapping[5], we use the prior
energy term to correct label rationality.
b b b
f f b
f f b
b
,
j i
i j i
r N
E D
r
r r r
. .if
otherwise
i p etrue E
change
false
r
max min min
. . . . . . . .0.5p e opposite p e p e p erate
45 CCU CSIE
44. POST-PROCESSING & CONFIDENCE MEASURE
Run Morphological operation Erosion on both
tracker results
Suitable for pixel-based methods
Suitable for those “fog-like” regions
Predict the location of the next ROI.
Combine both results
Use XOR operation to perform confidence
measurement
46 CCU CSIE
45. MANUAL REFINEMENT TOOL
To suitable for “fog-like”
regions, we implement a
brush-like tool to change
regions’ label easily.
Regions with more than 50%
percent areas are passed
through by the brush, will
modify labels.
The result after refinement will
be used to update trackers.
47 CCU CSIE
47. EXPERIMENTAL RESULTS
Environment:
Implemented in Visual C++
GUI implemented in ATL in Visual Studio 2005
Test sequence:
Use digital-video camera;
Resolution at 720 x 480 pixels;
Targets are larger than usual;
49 CCU CSIE
