Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Moving object detection

moving object detection using optical flow

  • Login to see the comments

Moving object detection

  1. 1. Moving Object Detection
  2. 2. Tracking Tracking is the problem of estimating the trajectory of an object in the image plane as it moves around a scene. 2 DSG, CEERI Pilani 12/8/2011
  3. 3. What is Motion Tracking…?• Technologies that collect data on human movement (input) used to control sounds, music, recorded or projected text, video art, stage lighting (output) via performer actions / gestures / movements / bio-data. 3 DSG, CEERI Pilani 12/8/2011
  4. 4. What is Motion Tracking…?• other uses: • Animation modeling (motion capture) • Scientific research (musicology, semantics, ergonomics, medicine, sports medicine, architecture) • Therapy for physically and mentally handicapped 4 DSG, CEERI Pilani 12/8/2011
  5. 5. Motion tracking vs. Motion captureMotion capture Motion tracking•Tracks location of fixed •less equipment, less data,positions on body •less cost ($1k-2k)•Highlyaccurate, expensive •concerned with motion($200k-2m) qualities like dynamic, direction of motion•Generally not realtime •real time•Used for data collection(research) and for making •used for live applications:human or animal motion in installationanimations art, dance, theater and(films, games, etc.) more5 music DSG, CEERI Pilani 12/8/2011
  6. 6. Motion capture Motion capture is defined as "The creation of a 3D representation of a live performance." Tracks location of fixed positions on body with reflective markers 12-24 cameras, each lens is ringed with infrared lights 6 DSG, CEERI Pilani 12/8/2011
  7. 7. Motion capture Motion capture is used to be considered a tool for creating animation. 7 DSG, CEERI Pilani 12/8/2011
  8. 8. Motion capture Typical uses  Human movement research (sports, musicology, ergonomics, medicine,...)  Film and Animation -- often used with 3-D animation (modeling) programs to make animations  maya (  houdini ( 8 DSG, CEERI Pilani 12/8/2011
  9. 9. Motion capture  Vicon is a leading company in motion capture9 DSG, CEERI Pilani 12/8/2011
  10. 10. Motion tracking media output sounds, musi input c, text, projec physical tions, lighting human action10 DSG, CEERI Pilani 12/8/2011
  11. 11. Motion tracking sensor output device (e.g. video computer (e.g. loud media output camera) speakers) input sounds, musi physical c, text, projechuman action tions, lighting analogue to digital to digital analogue conversion conversion11 DSG, CEERI Pilani 12/8/2011
  12. 12. Digital vs. Analogue Digital data Analogue data • easy to reproduce • hard to reproduce • "rich data" (infinite values) • lower resolution, less human- • very high resolution feel. • more details • easy to store • contaminated data (becomes • easy to process noisy, but rarely fails • contaminated data remains completely) clean (errors can be filtered) or signal fails altogether 12 DSG, CEERI Pilani 12/8/2011
  13. 13. What you need to track motion Computer needs video input Digital video (Firewire, USB2) + digital cameras (camcorder, webcams) + low noise + works with laptops - latency issues - image resolution issues (smaller chip sizes) - limited cable length Analog video + "unlimited" cable length + lower latency + even digital cams usually have analog output - cost more (although many older cameras work quite well) - works less well with laptops i.e. need an external or internal framegrabber13 DSG, CEERI Pilani 12/8/2011
  14. 14. Who is using motion tracking?  Palindrome Intermedia Performance Group  Krisztina de Chatel  Igloo  Ventura Dance (Pablo Ventura)  Robert Lepage  André Werner  Marlon Barrios Solano  La la la Human Steps  Georg Hobmeier  Leine Roebana Dans Kompanie  Troika Ranch  Blue Man Group  you14 DSG, CEERI Pilani 12/8/2011
  15. 15. The Problem in Motion Tracking  Given a set of images in time which are similar but not identical, derive a method for identifying the motion that has occurred (in 2d) between different images.
  16. 16. Motion Detection and Estimation in Literature Image differencing  based on the threshold difference of successive images  difficult to reconstruct moving areas Background subtraction  foreground objects result by calculating the difference between an image in the sequence and the background image (previously obtained)  remaining task: determine the movement of these foreground objects between successive frames Block motion estimation  Calculates the motion vector between frames for sub-blocks of the image  mainly used in image compression  too coarse
  17. 17. What Is Optical Flow? Optical flow is the displacement field for each of the pixels in an image sequence. For every pixel, a velocity vector dx , dy is found which says: dt dt  how quickly a pixel is moving across the image  the direction of its movement.
  18. 18. Optical Flow Examples Image #1 Optical Flow Field Image #218 DSG, CEERI Pilani 12/8/2011
  19. 19. Optical Flow Examples Translation Rotation Scaling
  20. 20. Estimation of the optical flow Sequences of ordered images allow the estimation of motion as either instantaneous image velocities or discrete image displacements. The optical flow methods try to calculate the motion between two image frames which are taken at times t and t + δt at every voxel position. 20 DSG, CEERI Pilani 12/8/2011
  21. 21. Voxel Position A voxel (volumetric pixel or Volumetric Picture Element) is a volume element, representing a value on a regular grid in three dimensional space. A series of voxels in a stack with a single voxel 21 highlighted DSG, CEERI Pilani 12/8/2011
  22. 22. Estimation of the optical flow Optical Flow methods are called differential since they are based on local Taylor series approximations of the image signal; that is, they use partial derivatives with respect to the spatial and temporal coordinates. In mathematics, a Taylor series is a representation of a function as an infinite sum of terms that are calculated from the values of the functions derivatives at a single point. 22 DSG, CEERI Pilani 12/8/2011
  23. 23. Taylor series  The Taylor series of a real or complex function ƒ(x) that is infinitely differentiable in a neighborhood of a real or complex number a is the power series which can be written in the more compact sigma notation aswhere n! denotes the factorial of n and ƒ (n)(a) denotes the nth derivative of ƒevaluated at the point a. The zeroth derivative of ƒ is defined to be ƒ itself and (x −a)0 and 0! are both defined to be 1. In the case that a = 0, the series is also calleda Maclaurin series. 23 DSG, CEERI Pilani 12/8/2011
  24. 24. Estimation of the optical flow For a 2D+t dimensional case (3D or n-D cases are similar) a voxel at location (x,y,t) with intensity I(x,y,t) will have moved by δx, δy and δt between the two image frames, and the following image constraint equation can be given: I(x,y,t) = I(x + δx,y + δy,t + δt) Assuming the movement to be small, the image constraint at I(x,y,t) with Taylor series can be developed to get: H.O.T (higher-order terms) 24 DSG, CEERI Pilani 12/8/2011
  25. 25. Estimation of the optical flow25 DSG, CEERI Pilani 12/8/2011
  26. 26. Estimation of the optical flow where Vx,Vy are the x and y components of the velocity or optical flow of I(x,y,t) and are the derivatives of the image at (x,y,t) in the corresponding directions. Ix,Iy and It can be written for the derivatives in the following. Thus: IxVx + IyVy = − It orThis is an equation in two unknowns and cannot be solved as such. This isknown as the aperture problem of the optical flow algorithms. To find theoptical flow another set of equations is needed, given by some additionalconstraint. All optical flow methods introduce additional conditions forestimating the actual flow. 26 DSG, CEERI Pilani 12/8/2011
  27. 27. Our Solution Optical flow: maximum one pixel large movements Optical flow: larger movements Morphological filter Contour detection (demo purposes)
  28. 28. Optical Flow: maximum one pixel largemovements The optical flow for a pixeli, j given 2 successive images and k : k 1 mk (i, j ) ( x, y) so that I k (i, j) I k 1 (i x, j y) (1) is minimum for 1 x 1, 1 y 1 k k+1
  29. 29. Optical Flow: maximum one pixellarge movements More precision: consider a 3×3 window around the pixel: Optical flow for pixel i, j becomes: mk (i, j ) ( x, y) so that 1 1 1 1 I k (i u, j v) I k 1 (i u x, j v y) (2) u 1v 1 u 1v 1 is minimum for 1 x 1, 1 y 1
  30. 30. Optical Flow: larger movements Reduce the size of the image => reduced size of the movement Solution: multi-resolution analysis of the images Advantages: computing efficiency, stability
  31. 31. Multi-resolution Analysis Coarse to fine optical flow estimation: 32 32 64 64 128 128 256 256 Original image k Original image k+1
  32. 32. Optical Flow: Top-down StrategyAlgorithm (1/4 scale of resolution reduction): Step 1: compute optical flow vectors for the highest level of the pyramid l (smallest resolution) Step 2: double the values of the vectors Step 3: first approximation: optical flow vectors for the (2i, 2j), (2i+1, 2j), (2i, 2j+1), (2i+1, 2j+1) pixels in the l- 1 level are assigned the value of the optical flow vector for the (i,j) pixel from the l level Level l Level l-1
  33. 33. Optical Flow: Top-down Strategy Step 4:  adjustment of the vectors of the l-1(one) level in the pyramid  method: detection of maximum one pixel displacements around the initially approximated position Step 5: smoothing of the optical flow field (Gaussian filter)
  34. 34. Filtering the Size of the Detected Regions Small isolated regions of motion detected by the optical flow method are classified as noise and are eliminated with the help of morphological operations: Step 1: Apply the opening: Step 2: Apply the  B X closing: B X B X B X B B
  35. 35. Contour Detection For demonstration purposes, the contours of the moving regions detected are outlined Method: the Sobel edge detector:  Compute the intensity gradient: f f f x, y , fx, f y (5) x y using the Sobel masks: 1 0 1 1 2 1 1 1 Gx 2 0 2 , Gy 0 0 0 (6) 4 4 1 0 1 1 2 1  Compute the magnitude of the gradient: M x, y f x, y fx 2 fy 2 (7)  if M x, y threshold edge pixel then else non-edge pixel
  36. 36. A Block Diagram of the System
  37. 37. Conclusions What we did:  We managed to estimate the motion with a certain level of accuracy  The results might be good enough for some applications, while other applications might require better accuracy What remains to be done:  Reduce computational complexity  use the computed background image to separate foreground objects  Parallelism of the algorithms  Experiment with specific problems, calibrate the parameters of the algorithms
  38. 38. Thanks38 DSG, CEERI Pilani 12/8/2011