Real-time Detection and Tracking of Multiple Objects with Partial Decoding in H.264/AVC Bitstream Domain Wonsang You Unive...
MOTIVATION <ul><li>Real-time Object Detection and Tracking in H.264|AVC Bitstream </li></ul>
Pixel Domain Approach <ul><li>Categories of Object Detection and Tracking Approaches. </li></ul><ul><ul><li>Pixel domain a...
Compressed Domain Approach <ul><li>Basic idea </li></ul><ul><ul><li>Exploit encoded information (DCT, motion vectors, etc)...
Related Works in Compressed Approach <ul><li>Basic Solution </li></ul><ul><ul><li>Using a low-resolution image from DCT co...
Our Solution for H.264/AVC Bitstreams <ul><li>Basic idea </li></ul><ul><ul><li>We use  partially-decoded pixel data  inste...
Overview of the Proposed Algorithm <ul><li>Extraction Phase </li></ul><ul><ul><li>Probabilistic Spatiotemporal Macroblock ...
EXTRACTION PHASE <ul><li>Real-time Object Detection and Tracking in H.264|AVC Bitstream </li></ul>
Probabilistic Spatiotemporal Macroblock Filtering <ul><li>Probabilistic Spatiotemporal Macroblock Filtering </li></ul><ul>...
Block Clustering <ul><li>Block clustering </li></ul><ul><ul><li>Removing skip macroblocks </li></ul></ul><ul><ul><li>Elimi...
Spatial Filtering <ul><li>Filtering block groups which are likely to be background </li></ul><ul><ul><li>Removing BGs of  ...
Temporal Filtering <ul><li>Filtering ABGs which are likely to be background </li></ul><ul><ul><li>Removing ABGs of backgro...
Temporal Filtering <ul><li>Observing occurrence of ABGs during a finite period </li></ul><ul><ul><li>ABGs with high occurr...
Temporal Filtering ABGs Criteria for survival of ABG as an object
REFINEMENT PHASE <ul><li>Real-time Object Detection and Tracking in H.264|AVC Bitstream </li></ul>
Background Subtraction in I-frames Reference Blocks (A-D) are substituted into background image Partial Decoding in I-fram...
Motion Interpolation in P-frames Assumption : The object moves slowly nearly with uniform motion in one GOP ROI Refinement...
Occlusion Handling Comparing Hue color histogram of two objects
Experimental Results (1/3) Indoor Sequence: 49.5 frames/sec Ourdoor Sequence: 37.12 frames/sec
Experimental Results (2/3)
Experimental Results (3/3)
Thank You! Wonsang You [email_address] University of Augsburg Germany
Upcoming SlideShare
Loading in …5
×

Real-time Object Tracking

2,838 views

Published on

Related article: Wonsang You, M.S. Houari Sabirin, and Munchurl Kim, "Real-time detection and tracking of multiple objects with partial decoding in H.264/AVC bitstream domain," Proceedings of SPIE, N. Kehtarnavaz and M.F. Carlsohn, San Jose, CA, USA: SPIE, 2009, pp. 72440D-72440D-12.

Published in: Technology, Economy & Finance
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,838
On SlideShare
0
From Embeds
0
Number of Embeds
13
Actions
Shares
0
Downloads
94
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Real-time Object Tracking

  1. 1. Real-time Detection and Tracking of Multiple Objects with Partial Decoding in H.264/AVC Bitstream Domain Wonsang You University of Augsburg, Germany Electronic Imaging, 19 January 2009
  2. 2. MOTIVATION <ul><li>Real-time Object Detection and Tracking in H.264|AVC Bitstream </li></ul>
  3. 3. Pixel Domain Approach <ul><li>Categories of Object Detection and Tracking Approaches. </li></ul><ul><ul><li>Pixel domain approach </li></ul></ul><ul><ul><li>Compressed domain approach </li></ul></ul><ul><li>Pixel domain approach. </li></ul><ul><ul><li>Using raw pixel data </li></ul></ul><ul><ul><li>High accuracy </li></ul></ul><ul><ul><li>High computational complexity </li></ul></ul><ul><ul><li>Require additional computation for compressed videos </li></ul></ul><ul><li>Compressed domain approach </li></ul><ul><ul><li>Exploit encoded information (DCT, motion vectors, etc) </li></ul></ul><ul><ul><li>Poor performance </li></ul></ul><ul><ul><ul><li>Applicable for simple scenarios </li></ul></ul></ul><ul><ul><ul><li>Weak for occlusion </li></ul></ul></ul>
  4. 4. Compressed Domain Approach <ul><li>Basic idea </li></ul><ul><ul><li>Exploit encoded information (DCT, motion vectors, etc) </li></ul></ul><ul><li>Advantages </li></ul><ul><ul><li>Remarkably fast processing time </li></ul></ul><ul><ul><li>Adaptive to compressed videos </li></ul></ul><ul><li>Disadvantages </li></ul><ul><ul><li>Unreliability of encoded information </li></ul></ul><ul><ul><li>Sparse assignment of block-based data </li></ul></ul><ul><ul><li>Poor performance </li></ul></ul><ul><ul><ul><li>Applicable for simple scenarios </li></ul></ul></ul><ul><ul><ul><li>Weak for occlusion </li></ul></ul></ul>
  5. 5. Related Works in Compressed Approach <ul><li>Basic Solution </li></ul><ul><ul><li>Using a low-resolution image from DCT coefficients </li></ul></ul><ul><ul><li>Unfortunately, impossible for AVC bitstreams </li></ul></ul>DC
  6. 6. Our Solution for H.264/AVC Bitstreams <ul><li>Basic idea </li></ul><ul><ul><li>We use partially-decoded pixel data instead of low-resolution images. </li></ul></ul><ul><li>Advantages </li></ul><ul><ul><li>Reliable performance in more natural scenes </li></ul></ul><ul><ul><ul><li>Articulated objects such as humans </li></ul></ul></ul><ul><ul><ul><li>Objects changing in size </li></ul></ul></ul><ul><ul><ul><li>Objects which have monotonous color or a chaotic set of motion vectors </li></ul></ul></ul><ul><ul><li>Occlusion handling </li></ul></ul><ul><ul><li>Detecting and tracking multiple objects in stationary background </li></ul></ul><ul><ul><li>Real-time processing </li></ul></ul><ul><ul><li>Partial decoding in I-frames </li></ul></ul><ul><ul><ul><li>It has been considered to be impossible </li></ul></ul></ul><ul><ul><ul><li>Due to spatial prediction dependency on neighboring blocks </li></ul></ul></ul>
  7. 7. Overview of the Proposed Algorithm <ul><li>Extraction Phase </li></ul><ul><ul><li>Probabilistic Spatiotemporal Macroblock Filtering </li></ul></ul><ul><ul><li>Roughly extracting the block-level region of objects </li></ul></ul><ul><ul><li>Constructing the approximate object trajectories in each P-frame </li></ul></ul><ul><li>Refinement Phase </li></ul><ul><ul><li>Accurately refining the obect trajectories </li></ul></ul><ul><ul><li>Background subtraction and partial decoding in I-frames </li></ul></ul><ul><ul><li>Motion interpolation in P-frames </li></ul></ul>
  8. 8. EXTRACTION PHASE <ul><li>Real-time Object Detection and Tracking in H.264|AVC Bitstream </li></ul>
  9. 9. Probabilistic Spatiotemporal Macroblock Filtering <ul><li>Probabilistic Spatiotemporal Macroblock Filtering </li></ul><ul><ul><li>Block-based filtering of background parts (BGs) </li></ul></ul><ul><ul><li>By using spatial and temporal properties of macroblocks </li></ul></ul><ul><ul><li>Rapid processing of segmenting object regions and tracking each object </li></ul></ul>
  10. 10. Block Clustering <ul><li>Block clustering </li></ul><ul><ul><li>Removing skip macroblocks </li></ul></ul><ul><ul><li>Eliminating probable background parts </li></ul></ul><ul><ul><li>Clustering the remaining MBs into several fragments </li></ul></ul><ul><li>Block group (BG) </li></ul><ul><ul><li>Set of non-skip blocks </li></ul></ul>BGs
  11. 11. Spatial Filtering <ul><li>Filtering block groups which are likely to be background </li></ul><ul><ul><li>Removing BGs of </li></ul></ul><ul><ul><ul><li>One-macroblock </li></ul></ul></ul><ul><ul><ul><li>All zero IT coefficients </li></ul></ul></ul><ul><li>Active Block Group (ABG) </li></ul><ul><ul><li>Remaining BGs after spatial filtering </li></ul></ul>ABGs : Remaining BGs after Spatial Filtering
  12. 12. Temporal Filtering <ul><li>Filtering ABGs which are likely to be background </li></ul><ul><ul><li>Removing ABGs of background </li></ul></ul><ul><ul><ul><li>Based on temporal consistency of each ABG over a given period </li></ul></ul></ul><ul><ul><li>Fragments with high occurrence probability: considered as a part of objects </li></ul></ul>Remaining ABGs after Temporal Filtering
  13. 13. Temporal Filtering <ul><li>Observing occurrence of ABGs during a finite period </li></ul><ul><ul><li>ABGs with high occurrence for finite period are judged as &quot;Real Object&quot;. </li></ul></ul><ul><ul><li>Occurrence Probability is measured. </li></ul></ul>ABGs
  14. 14. Temporal Filtering ABGs Criteria for survival of ABG as an object
  15. 15. REFINEMENT PHASE <ul><li>Real-time Object Detection and Tracking in H.264|AVC Bitstream </li></ul>
  16. 16. Background Subtraction in I-frames Reference Blocks (A-D) are substituted into background image Partial Decoding in I-frames ROI Refinement in I-frames A B C D
  17. 17. Motion Interpolation in P-frames Assumption : The object moves slowly nearly with uniform motion in one GOP ROI Refinement in P-frames In the ROI prediction stage, ROI significantly vary over P-frames. So, ROI refinement is needed for P-frames.
  18. 18. Occlusion Handling Comparing Hue color histogram of two objects
  19. 19. Experimental Results (1/3) Indoor Sequence: 49.5 frames/sec Ourdoor Sequence: 37.12 frames/sec
  20. 20. Experimental Results (2/3)
  21. 21. Experimental Results (3/3)
  22. 22. Thank You! Wonsang You [email_address] University of Augsburg Germany

×