4 4 bopardikar_concealment


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

4 4 bopardikar_concealment

  1. 1. Structural Alignment-Based Temporal Concealment of Packet-Loss in Video Ajit S. Bopardikar, Odd Inge Hillestad and Andrew Perkis Centre for Quantifiable Quality of Service in Communication Systems Trondheim, Norway.
  2. 2. Publications Based On This Work <ul><li>A. S. Bopardikar, O. I. Hillestad and A. Perkis, “Temporal error concealment algorithm based on structural alignment for packet video,” International Conference on Signal Processing and Communications (SPCOM) , Bangalore, India, December 2004. </li></ul><ul><li>A. S. Bopardikar, O. I. Hillestad and A. Perkis, ”Structural alignment-based temporal concealment of packet-loss in video” in review, International Conference on Acoustics Speech and Signal Processing , Philadelphia, PA, USA, May 2005. </li></ul>
  3. 3. This Presentation Is About…. <ul><li>A novel temporal algorithm to conceal packet-loss related distortion in video. </li></ul><ul><li>Algorithm based on the insight that the spatial region around the area in the previous frame that yields the best concealment will in general be highly correlated with the corresponding area around the region to be concealed in the present frame. </li></ul>
  4. 4. Overview <ul><li>Packetization overview </li></ul><ul><li>Packet-loss related artifacts in block-based video compression schemes. </li></ul><ul><li>Spatial and temporal concealment methods. </li></ul><ul><li>BMA-type temporal concealment methods. </li></ul><ul><li>Proposed method. </li></ul><ul><li>Results and discussions. </li></ul>
  5. 5. Flowgraph for Video Stream Packetization Video stream RTP/UDP IP packets Internet/ Simulation Test net Smart mapping Flows File format
  6. 6. Example: MPEG-2 Bitstream
  7. 7. MPEG-2 Bitstream <ul><li>Each packet can be decoded independently irrespective of the errors in decoding previous packets. </li></ul><ul><li>Loss of packet means loss of spatial and motion vector information for that part of the frame. </li></ul>
  8. 8. Packet-loss Related Artifacts in Block-Based Coding Schemes
  9. 9. Concealment Methods for Packet-Loss Related Errors in Video <ul><li>Spatial </li></ul><ul><li>Temporal </li></ul>
  10. 10. Spatial Concealment Methods <ul><li>Bilinear interpolation based methods. </li></ul><ul><li>Projection onto convex sets (POCS). </li></ul><ul><li>Requires no knowledge of the previous frame(s). </li></ul><ul><li>Simple and can cause blurring. </li></ul><ul><li>Better results can be obtained using temporal methods. </li></ul>
  11. 11. Temporal Concealment Methods <ul><li>Makes use of the high degree of correlation that generally exists between consecutive frames. </li></ul><ul><li>The idea is conceal artifacts using ’best-fitting’ parts from the previous frame which is assumed to be error-free. </li></ul><ul><li>Examples: </li></ul><ul><ul><li>Temporal Replacement (TR) </li></ul></ul><ul><ul><li>Boundary Matching Algorithm (BMA). </li></ul></ul>
  12. 12. Temporal Replacement <ul><li>Replaces lost and damaged macroblocks (MBs) in the present frame with MBs from the same spatial location in the previous, error-free frame. </li></ul><ul><li>In presence of significant motion, TR can give rise to visible discontinuities at boundaries of replaced MBs. </li></ul>
  13. 13. Boundary Matching Algorithm <ul><li>Motion vector information typically lost with the packet. </li></ul><ul><li>These motion vectors need to be estimated for effective concealment. </li></ul><ul><li>BMA assembles a set of candidate motion vectors and then chooses a best fit from among the MBs in the previous frame associated with them. </li></ul><ul><li>Best fit MB minimizes the sum of mean squared error (MSE) or the sum of absolute difference (SAD) between its boundaries and the boundaries adjacent to them from the top, bottom and left MBs around the area to be concealed. </li></ul>Candidate MB
  14. 14. Proposed Algorithm <ul><li>Based on the high degree of correlation that can exist between spatial region around the best-fit candidate MB in the previous frame and the corresponding region around the MB-area to be concealed in the present frame. </li></ul>
  15. 15. Proposed Algorithm <ul><li>Insert candidate MB (M -1C ) in the area to be concealed. </li></ul><ul><li>Compute x and y Sobel gradient maps for the two composite blocks. </li></ul><ul><li>Compute the overall magnitude gradient map. </li></ul><ul><li>In the gradient map, compute the SAD between the corresponding red and green boundaries of the surrounding MBs. </li></ul><ul><li>Choose the MB from the previous frame that minimizes the sum of the three SADs for concealment. </li></ul>
  16. 16. Simulations: Particulars <ul><li>Full search adopted to search for the best fitting MB. </li></ul><ul><li>The area in the previous frame for the search is 52x52 pixels centered on the spatial position of the MB to be concealed. </li></ul><ul><li>All video clips in SIF (352x240 pixels, 30 frames per second) MPEG-2 coded at 1.5Mbps. </li></ul><ul><li>Packet-loss simulated using NTT DoCOMo software. PLR = 0.001. </li></ul><ul><li>Performance compared to a BMA-like algorithm. </li></ul>
  17. 17. Simulations: Results for Frame 346 of ’Table Tennis’ sequence. Original PL ( 13.84 dB) BMA (28.91 dB) Proposed (36.34 dB)
  18. 18. Simulation Results: PSNR 29.31 25.98 15.72 325 32.93 28.05 14.89 304 Table-tennis 30.47 27.30 14.32 208 28.75 19.99 8.92 67 26.64 19.35 8.96 46 Flower Garden 39.07 24.94 14.02 202 30.17 22.65 13.19 139 29.58 27.13 21.93 72 Mobile 39.42 28.29 10.55 208 31.07 29.35 11.17 120 39.91 38.68 17.53 67 Susie Proposed (dB) BMA (dB) PL (dB) Fr. no. Sequence
  19. 19. Conclusions <ul><li>Proposed algorithm better at aligning edges. </li></ul><ul><li>Relies on matching the structure around the MB to be concealed and the candidate MB. </li></ul><ul><li>Low computational complexity. </li></ul>