This document discusses techniques for segmenting independently moving image regions using motion detection. It covers the following approaches: 1. Motion-based segmentation using optical flow to detect pixel-level motion between frames. This approach has limitations due to the aperture and occlusion problems. 2. Color-based and texture-based segmentation which learn background models (e.g. histograms or Gaussian distributions) for each pixel and detect foreground objects that differ significantly from the background models. 3. Dominant motion segmentation fits a single motion model to partition a frame into regions of global and local motion. Multiple motion segmentation estimates multiple motion models competing at each pixel.

1. Labeling independently moving image regions

2. Motion Segmentation
 Foreground (object) and Background (noise)
Result could be a
 Binary image, containing foreground only
 Probability image, containing the likelihood of each pixel being
foreground
 Approaches
 Motion-based (optical flow)
 Color-based
 Texture-based
2

3. Motion (Change) Detection
 Static camera : changed and unchanged regions.
 Moving camera : global and local motion regions.
Limitations associated with motion estimation
 Aperature problem : pixels in a flat image region may
appear stationary even if they are moving as a result of
an aperture problem (hence the need for hierarchical
methods)
 Occlusion Problem : erroneous labels may be
assigned to pixels in covered or uncovered image
regions as a result of an occlusion problem.

4. Motion Detection: using two frames :
Image Differencing

5. 5
Subtract Image
 Compute pixel-wise
 Subtract previous image from input image:
 Usually the absolute distance is applied
),(),(),( yxByxIyxF 
),( yxF
1. Save image in last frame
2. Capture camera image
3. Subtract image
4. Threshold
5. Delete noise

6. 1
2
3
4
5
6
7
8
1
2
5
4
3
6
7
8
9
10

7. 7
Threshold
 Decide, when a pixel is supposed to be considered
as a background pixel, or when it is to be
considered as a foreground pixel:
 Pixel is foreground pixel, if
 Pixel is background pixel, if
 Problem: What TH?!?
),( yx
THyxF ),(
),( yx
THyxF ),(
1. Save image in last frame
2. Capture camera image
3. Subtract image
4. Threshold
5. Delete noise

8. 1
2
3
4
5
6
7
8

9. 125 4 3678
Frame
no. 1 2 3 4 5 6 7 8 9
x 154 160 160 160 157 130 112 79 0
y 13 19 19 19 16 19 19 19 0
w s s w w w w w

10. Dominant Motion : no optical flow
 Spatio temporal intensity gradient.
 Dominant motion segmentation
 fitting a single parametric motion model
 partition the frame in two pixel groups
 Repeat step 1 only to well represented pixels group.

11. Thresholding
Noise Removal Median Filter
Image subtraction
Bounding Box
Centroid

12. Multiple Motion Segmentation
 Multiple motion models compete against each other at
each decision site. They consist estimating:
 Motion within each region (motion model)
 spatial support of each region
 number of regions.
 The problem : associate each pixel to the right motion
model, while simultaneously estimating motions and
supports.
 Clustering (K-means, hough transform),
 Maximum Likelihood (ML) (pixel based) and
 Maximum A Posteriori probability (MAP).
 Region based label assignment

13. Optical flow estimation
motion vectors at each frame
Thresholding
Morphological closing on the motion vectors

14. Object tracking using kalman filters (prediction)

15. 15
Motion Model
 Predicted position at time t:
 Brownian Motion: According to a Gaussian model
 0’th order:
 1’th order:
 Similar for y
 2’th order
 Similar for y
),( tt yx
),(),( 11  tttt yxyx
1at timeinvelocity:1
11
t-xx
xtxx
t
ttt

 


1at timeinonaccelerati:
2
1
1
11
2
1
t-xx
xtxtxx
t
tttt

 



16. 16
Color Based segmentation : Background
Subtraction
 Use Neighborhood relation!!
 Compare pixel with its neighbors!!
 Weight them!!
 Learn the background and its variations!!
E.g. Gaussian models (mean,var) for each pixel!!!
E.g. a Histogram for each Pixel
 The more images you train on the better!!
 Algorithm:
 Consider each pixel (x,y) in the input image and check, how much it
varies with respect to the mean and variance of the learned Gaussian
models?
1. Calculate mean and variance for each pixel
2. Capture camera image
3. Subtract image (= motion)
4. Weight the distances (new)
5. Threshold according to variance
6. Delete noise

17. Color Segmentation with Histograms

18. Color Segmentation with Histograms
18
brightness

19. 19
Color Segmentation with
Gaussian Distribution
N(m, C)