Background subtraction


Published on

its about background subtraction and also to detect moving object

Published in: Technology
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Background subtraction

  1. 1. Background Subtraction12/8/2011 1
  2. 2. Background Subtraction• As the name suggests, background subtraction is the process of separating out foreground objects from the background in a sequence of video frames.12/8/2011 2
  3. 3. Background Subtraction• Background subtraction is a widely used approach for detecting moving objects from static cameras.12/8/2011 3
  4. 4. Fundamental Logic• Fundamental logic for detecting moving objects from the difference between the current frame and a reference frame, called “background image” and this method is known as FRAME DIFFERENCE METHOD12/8/2011 4
  5. 5. Background Modelling12/8/2011 5
  6. 6. Background Model12/8/2011 6
  7. 7. After Background Filtering…12/8/2011 7
  8. 8. Background Subtraction PrinciplesAt ICCV 1999, MS Research presented a study, Wallflower: Principles and Practiceof Background Maintenance, by Kentaro Toyama, John Krumm, BarryBrumitt, Brian Meyers. This paper compared many different backgroundsubtraction techniques and came up with some principles: P1: P2: P3: P4: 12/8/2011 8
  9. 9. The Problem - RequirementsThe background image is not fixed but must adapt to:• Illumination changes •Gradual •Sudden (such as clouds)12/8/2011 9
  10. 10. • Motion changes •Camera oscillations • High-frequencies background objects (such as tree branches, sea waves, and similar)12/8/2011 10
  11. 11. • Changes in the background geometry •Parked cars12/8/2011 11
  12. 12. Widely Used!• Traffic monitoring (counting vehicles, detecting & tracking vehicles),• Human action recognition (run, walk, jump, squat),• Human-computer interaction (“human interface")• Object tracking (watched tennis lately),• And in many other cool applications of computer vision such as digital forensics.12/8/2011 12
  13. 13. Simple Approach Image at time t: Background at time t: I (x, y, t) B(x, y, t) > Th1. Estimate the background for time t.2. Subtract the estimated background from the input frame.3. Apply a threshold, Th, to the absolute difference to get theforeground mask.12/8/2011 13
  14. 14. The basic methods• Frame difference: | framei–framesi-1| > Th• The estimated background is just the previous frame• It evidently works only in particular conditions of objects’ speed and frame rate• Very sensitive to the threshold Th12/8/2011 14
  15. 15. Frame Differencing• Background is estimated to be the previous frame. Background subtraction equation then becomes: B(x ,y ,t) = I (x ,y ,t-1) I (x, y, t) - I (x, y, t – 1) > Th12/8/2011 15
  16. 16. Frame Differencing• Depending on the object structure, speed, frame rate and global threshold, this approach may or may not be useful (usually not). > Th12/8/2011 16
  17. 17. Frame Differencing Th = 25 Th = 50 Th = 100 Th = 20012/8/2011 17
  18. 18. Thresholding• The key parameter in the thresholding process is the choice of the threshold value.• Several different methods for choosing a threshold exist, – Users can manually choose a threshold value, or – A thresholding algorithm can compute a value automatically, which is known as automatic thresholding12/8/2011 18
  19. 19. Motivation• A robust background subtraction algorithm should handle: lighting changes, repetitive motions from clutter and long- term scene changes.12/8/2011 19
  20. 20. Algorithm Overview• The values of a particular pixel is modeled as a mixture of adaptive Gaussians. – Why mixture? Multiple surfaces appear in a pixel. – Why adaptive? Lighting conditions change.• At each iteration Gaussians are evaluated using a simple heuristic to determine which ones are mostly likely to correspond to the background.• Pixels that do not match with the “background Gaussians” are classified as foreground.• Foreground pixels are grouped using 2D connected component analysis.12/8/2011 20
  21. 21. Advantages vs. Shortcomings• Advantages: – A different “threshold” is selected for each pixel. – These pixel-wise “thresholds" are adapting by time. – Objects are allowed to become part of the background without destroying the existing background model. – Provides fast recovery.• Disadvantages: – Cannot deal with sudden, drastic lighting changes. – Initializing the Gaussians is important (median filtering). – There are relatively many parameters, and they should be selected intelligently.12/8/2011 21
  22. 22. Challenges background subtraction Algorithm• It must be robust against changes in illumination.• It should avoid detecting non-stationary background objects such as swinging leaves, rain, snow ,and shadow cast by moving objects.• Finally, its internal background model should react quickly to changes in background such as starting and stopping of vehicles.12/8/2011 22
  23. 23. BACKGROUND SUBTRACTION ALGORITHMS Flow diagram of a generic background subtraction algorithm.12/8/2011 23
  24. 24. Background subtraction Algorithms• The four major steps in a background subtraction algorithm are: 1. Preprocessing, 2. Background modeling, 3. Foreground detection, and 4. Data validation.12/8/2011 24
  25. 25. Background Modeling• Background modeling is at the heart of any background subtraction algorithm.• Background model is that which robust against environmental changes in the background, but sensitive enough to identify all moving objects of interest.12/8/2011 25
  26. 26. Background Modeling• We classify background modeling techniques into two broad categories – Non-recursive and – Recursive12/8/2011 26
  27. 27. Non-recursive Techniques• A non-recursive technique uses a sliding window approach for background estimation.• Non-recursive techniques are highly adaptive as they do not depend on the history beyond those frames stored in the buffer.12/8/2011 27
  28. 28. Non-recursive Techniques• Some of the commonly-used non-recursive techniques are:- – Frame differencing – Median filter – Mean filter12/8/2011 28
  29. 29. Frame differencingThis we have studied in earlier slide12/8/2011 29
  30. 30. Median filter• The background estimate is defined to be the median at each pixel location of all the frames in the buffer.• The assumption is that the pixel stays in the background for more than half of the frames in the buffer.12/8/2011 30
  31. 31. Median Filter• Assuming that the background is more likely to appear in a scene, we can use the median of the previous n frames as the background model:12/8/2011 31
  32. 32. 12/8/2011 32
  33. 33. 12/8/2011 33
  34. 34. Mean Filter• In this case the background is the mean of the previous n frames:12/8/2011 34
  35. 35. For n = 10: Estimated Background Foreground Mask For n = 20: Estimated Background Foreground Mask12/8/2011 35
  36. 36. For n = 50: Estimated Background Foreground Mask12/8/2011 36
  37. 37. Kalman filter• Kalman filter is a widely-used recursive technique for tracking linear dynamical systems under Gaussian noise. Many different versions have been proposed for background modeling, differing mainly in the state spaces used for tracking.12/8/2011 37
  38. 38. Mixture of Gaussians (MoG)• Unlike Kalman filter which tracks the evolution of a single Gaussian, the MoG method tracks multiple Gaussian distributions simultaneously.• MoG maintains a density function for each pixel.• It is capable of handling multi-modal background distributions.• Since MoG is parametric, the model parameters can be adaptively updated without keeping a large buffer of video frames.12/8/2011 38
  39. 39. Mixture of Gaussians • the pixel process of a pixel Z is the set of the last t values at the pixel • {Z0, Z1, ..., Zt−1}12/8/2011 39
  40. 40. Gaussian Mixture Model (GMM) • A GMM is a mixture pdf which is a linear combination of K Gaussian pdfs. • wi = 1 • each pixel is given one GMM12/8/2011 40
  41. 41. Background/Foreground Detection• There are two types of B/F detection methods: – Non-adaptive: It depend on certain numbers of video frames and do not maintain a background model in the algorithm. • B/F detection based on two frames • B/F detection based on three frames – Adaptive: It maintain a background model and the parameters of the background model evolve over time. • Adaptive B/F detection Statistical • B/F detection based on Gaussian model12/8/2011 41
  42. 42. Drawback of non-adaptive methods• The non-adaptive methods are useful only in – high-supervised, – short-term tracking applications without significant changes in the video scene. – When errors happen, it requires manual re- initialization. – Without re-initialization, errors in the background accumulate over time.12/8/2011 42
  43. 43. Adaptive Background/Foreground Detection • Adaptive B/F detection is chosen for more and more VCA applications because of the limitation of non-adaptive methods. A standard adaptive B/F detection method maintains a background model within the system.12/8/2011 43
  44. 44. Video Content Analysis (VCA) is the capability of automatically analyzing video to detect and determine temporal events not based on a single image12/8/2011 44
  45. 45. The algorithm of adaptive B/F detection• The algorithm of adaptive B/F detection is described below. – fi : A pixel in a current frame, where i is the frame index. – µ : A pixel of the background model (fi and m are located at the same location). – di: Absolute difference between fi and m. – bi: B/F mask - 0: background. 0 x ff: foreground. – T: Threshold – α: Learning rate of the background.12/8/2011 45
  46. 46. Adaptive B/F Detection1. di = |fi - µ|2. If di > T , fi belongs to the foreground; otherwise, it belongs to the background. Adaptive B/F Detection12/8/2011 46
  47. 47. Examples of Adaptive B/F Detection Examples of Adaptive B/F Detection12/8/2011 47
  48. 48. Demo12/8/2011 48
  49. 49. THANK YOU12/8/2011 49