Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A Comparison of Block-Matching Motion Estimation Algorithms

8,950 views

Published on

  • Be the first to comment

A Comparison of Block-Matching Motion Estimation Algorithms

  1. 1. A Comparison of Block-Matching Motion Estimation Algorithms María Santamaría and María Trujillo October 4th 2012 Séptimo Congreso Colombiano de Computación, 7CCC 2012, Medellín - Colombia
  2. 2. Multimedia and Vision Laboratory  MMV is a research group of the Universidad del Valle in Cali, Colombia M. Santamaría M. Trujillo & Computer Vision 3D World Optics Problem Camera Inverse System Problem 2D ImagesMultimedia and Vision Research Laboratory: http://mmv-lab.univalle.edu.co A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 2
  3. 3. Content Motivation Motion Estimation Block-Matching  Distortion Metrics  Selected Algorithms Evaluation  Quality Metrics  Performance Metrics Video Test Sequences Results Final Remarks A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 3
  4. 4. Motivation Video coding Tracking 3D TV Gesture recognition Resolution enhancementhttp://www.encodedmedia.com/http://assets.vr-zone.net/15416/LGTV.jpghttp://csecar.wordpress.com/http://www.newelectronics.co.uk/electronics-news/qualcomm-invests-in-gesture-recognition-technology/35620/http://users.soe.ucsc.edu/~milanfar/research/resolution-enhancement.html A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 4
  5. 5. Motion Estimation Video Frames Motion Estimation Motion VectorsA Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 5
  6. 6. Block-Matching Reference Frame Current FrameSearch areaCurrent blockBest matched blockMotion vectorA Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 6
  7. 7. Distortion Metrics  The two most popular measures to determine the match between two blocks are: the Mean Square Error (MSE) and the Sum of Absolute Differences (SAD) Distortion x yB. Xiong and C. Zhu, “A new multiplication-free block matching criterion,” IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 10, 2008Elliot J. Rouse. A virtual curriculum vitae. http://www.elliottjrouse.com/ A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 7
  8. 8. Full-Search (FS)  The Full-Search algorithm evaluates all positions in the window search of (2W+1) x (2W+1) size  It involves high computational cost  It is simple  It guarantees a high accuracy in finding the best match 1st stage Best matchedY. Huzka, and P. Kulla, “Trends in Block-matching Motion Estimation Algorithms,” 2004 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 8
  9. 9. Three-Step Search (3SS) Search centre 1st stageT. Koga, K. Iinuma, A. Hirano, Y. Iijima, and T. Ishiguro, “Motion Compensated Interframe Coding for Video Conferencing,” Proc. Nat.Telcommun. Conf., 1981 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 9
  10. 10. Three-Step Search (3SS) Search centre 1st stage 2nd stage Best candidateT. Koga, K. Iinuma, A. Hirano, Y. Iijima, and T. Ishiguro, “Motion Compensated Interframe Coding for Video Conferencing,” Proc. Nat.Telcommun. Conf., 1981 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 10
  11. 11. Three-Step Search (3SS) Search centre 1st stage 2nd stage 3rd stage Best candidateT. Koga, K. Iinuma, A. Hirano, Y. Iijima, and T. Ishiguro, “Motion Compensated Interframe Coding for Video Conferencing,” Proc. Nat.Telcommun. Conf., 1981 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 11
  12. 12. Three-Step Search (3SS)  The number of stages depends on the initial distance to which the first 9 neighbors are selected Search centre 1st stage 2nd stage 3rd stage Best matchedT. Koga, K. Iinuma, A. Hirano, Y. Iijima, and T. Ishiguro, “Motion Compensated Interframe Coding for Video Conferencing,” Proc. Nat.Telcommun. Conf., 1981 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 12
  13. 13. Four-Step Search (4SS) Search centre 1st stageL.-M. Po, and W. C.-Ma, “A novel four-step search algorithm for fast block motion estimation,” IEEE Trans. Circuits Syst. Video Technol., vol. 6,no. 3, 1996 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 13
  14. 14. Four-Step Search (4SS) Search centre 1st stage 2nd stage Best candidateL.-M. Po, and W. C.-Ma, “A novel four-step search algorithm for fast block motion estimation,” IEEE Trans. Circuits Syst. Video Technol., vol. 6,no. 3, 1996 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 14
  15. 15. Four-Step Search (4SS) Search centre 1st stage 2nd stage 3rd stage Best candidateL.-M. Po, and W. C.-Ma, “A novel four-step search algorithm for fast block motion estimation,” IEEE Trans. Circuits Syst. Video Technol., vol. 6,no. 3, 1996 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 15
  16. 16. Four-Step Search (4SS) Search centre 1st stage 2nd stage 3rd stage 4th stage Best candidateL.-M. Po, and W. C.-Ma, “A novel four-step search algorithm for fast block motion estimation,” IEEE Trans. Circuits Syst. Video Technol., vol. 6,no. 3, 1996 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 16
  17. 17. Four-Step Search (4SS)  Each new stage (except the reduced step stage) evaluates three or five blocks Search centre 1st stage 2nd stage 3rd stage 4th stage Best matchedL.-M. Po, and W. C.-Ma, “A novel four-step search algorithm for fast block motion estimation,” IEEE Trans. Circuits Syst. Video Technol., vol. 6,no. 3, 1996 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 17
  18. 18. Diamond Search (DS) Search centre 1st stageJ. Y. Tham, S. Ranganath, M. Ranganath, and A. A. Kassim, “A novel unrestricted center-biased diamond search algorithm for block motionestimation,” IEEE Trans. Circuits Syst. Video Technol., vol. 8, no. 4, 1998 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 18
  19. 19. Diamond Search (DS) Search centre 1st stage 2nd stage Best candidateJ. Y. Tham, S. Ranganath, M. Ranganath, and A. A. Kassim, “A novel unrestricted center-biased diamond search algorithm for block motionestimation,” IEEE Trans. Circuits Syst. Video Technol., vol. 8, no. 4, 1998 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 19
  20. 20. Diamond Search (DS) Search centre 1st stage 2nd stage 3rd stage Best candidateJ. Y. Tham, S. Ranganath, M. Ranganath, and A. A. Kassim, “A novel unrestricted center-biased diamond search algorithm for block motionestimation,” IEEE Trans. Circuits Syst. Video Technol., vol. 8, no. 4, 1998 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 20
  21. 21. Diamond Search (DS) Search centre 1st stage 2nd stage 3rd stage 4th stage Best candidateJ. Y. Tham, S. Ranganath, M. Ranganath, and A. A. Kassim, “A novel unrestricted center-biased diamond search algorithm for block motionestimation,” IEEE Trans. Circuits Syst. Video Technol., vol. 8, no. 4, 1998 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 21
  22. 22. Diamond Search (DS)  Each new stage (except the reduced step stage) evaluates four or five blocks  The neighbors are selected at a mixed distance Search centre 1st stage 2nd stage 3rd stage 4th stage Best matchedJ. Y. Tham, S. Ranganath, M. Ranganath, and A. A. Kassim, “A novel unrestricted center-biased diamond search algorithm for block motionestimation,” IEEE Trans. Circuits Syst. Video Technol., vol. 8, no. 4, 1998 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 22
  23. 23. Hexagonal Block Search (HEXBS) Search centre 1st stageC.-H. Cheung and L.-M. Po, “Novel cross-diamond-hexagonal search algorithms for fast block motion estimation,” IEEE Trans. Multimedia,vol. 7, no. 1, 2005 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 23
  24. 24. Hexagonal Block Search (HEXBS) Search centre 1st stage 2nd stage Best candidateC.-H. Cheung and L.-M. Po, “Novel cross-diamond-hexagonal search algorithms for fast block motion estimation,” IEEE Trans. Multimedia,vol. 7, no. 1, 2005 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 24
  25. 25. Hexagonal Block Search (HEXBS) Search centre 1st stage 2nd stage 3rd stage Best candidateC.-H. Cheung and L.-M. Po, “Novel cross-diamond-hexagonal search algorithms for fast block motion estimation,” IEEE Trans. Multimedia,vol. 7, no. 1, 2005 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 25
  26. 26. Hexagonal Block Search (HEXBS) Search centre 1st stage 2nd stage 3rd stage 4th stage Best candidateC.-H. Cheung and L.-M. Po, “Novel cross-diamond-hexagonal search algorithms for fast block motion estimation,” IEEE Trans. Multimedia,vol. 7, no. 1, 2005 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 26
  27. 27. Hexagonal Block Search (HEXBS)  Each new stage (except the reduced step stage) evaluates three blocks  It is faster than the DS, but has a lower quality of prediction Search centre 1st stage 2nd stage 3rd stage 4th stage Best matchedC.-H. Cheung and L.-M. Po, “Novel cross-diamond-hexagonal search algorithms for fast block motion estimation,” IEEE Trans. Multimedia,vol. 7, no. 1, 2005 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 27
  28. 28. Multi-Directional Gradient Descent Search (MDGDS) Search centre 1 1st stage 1 1 8 1 2 7 3 3 3 6 5 4 4 4L.-M. Po, K.-H. Ng, K.-M. Wong, and K.-W. Cheung, “Multi-direction search algorithm for block-based motion estimation,” in IEEE Asia PacificConf. in Circuits and Systems (APPCAS), 2008 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 28
  29. 29. Multi-Directional Gradient Descent Search (MDGDS) 6 1 Search centre 6 1 1st stage 5 5 5 2 2 2 2 2 2nd stage 4 3 Best candidateL.-M. Po, K.-H. Ng, K.-M. Wong, and K.-W. Cheung, “Multi-direction search algorithm for block-based motion estimation,” in IEEE Asia PacificConf. in Circuits and Systems (APPCAS), 2008 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 29
  30. 30. Multi-Directional Gradient Descent Search (MDGDS) 1 Search centre 6 1 2 1st stage 2nd stage 5 4 3 3rd stage 4 3 Best candidate 4 4L.-M. Po, K.-H. Ng, K.-M. Wong, and K.-W. Cheung, “Multi-direction search algorithm for block-based motion estimation,” in IEEE Asia PacificConf. in Circuits and Systems (APPCAS), 2008 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 30
  31. 31. Multi-Directional Gradient Descent Search (MDGDS) Search centre 1st stage 2nd stage 3rd stage 5 1 4th stage 2 Best candidate 4 3L.-M. Po, K.-H. Ng, K.-M. Wong, and K.-W. Cheung, “Multi-direction search algorithm for block-based motion estimation,” in IEEE Asia PacificConf. in Circuits and Systems (APPCAS), 2008 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 31
  32. 32. Multi-Directional Gradient Descent Search (MDGDS)  It tries to solve the problem of being trapped in a local minimum Search centre 1st stage 2nd stage 3rd stage 4th stage Best matchedL.-M. Po, K.-H. Ng, K.-M. Wong, and K.-W. Cheung, “Multi-direction search algorithm for block-based motion estimation,” in IEEE Asia PacificConf. in Circuits and Systems (APPCAS), 2008 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 32
  33. 33. Fast Directional Gradient Descent Search (FDGDS)  It is an improvement of the MDGDS that increases the speed of the algorithm and leads to little loss in quality of 1 prediction Search centre 1 1st stage 1 1 2 3 3 3 4 4 Relative Distortion Ratio 4 4L.-M. Po, K.-H. Ng, K.-W. Cheung, K.-M. Wong, Y. Uddin, and C.-W. Ting, “Novel Directional Gradient Descent Searches for Fast Block MotionEstimation,” IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 8, 2009 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 33
  34. 34. Fast Directional Gradient Descent Search (FDGDS) Search centre 1st stage 2nd stage Best candidate 1 2 1 2 3 3 3 3L.-M. Po, K.-H. Ng, K.-W. Cheung, K.-M. Wong, Y. Uddin, and C.-W. Ting, “Novel Directional Gradient Descent Searches for Fast Block MotionEstimation,” IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 8, 2009 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 34
  35. 35. Fast Directional Gradient Descent Search (FDGDS) Search centre 1st stage 2nd stage 3rd stage Best candidate 6 1 2 5 4 3L.-M. Po, K.-H. Ng, K.-W. Cheung, K.-M. Wong, Y. Uddin, and C.-W. Ting, “Novel Directional Gradient Descent Searches for Fast Block MotionEstimation,” IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 8, 2009 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 35
  36. 36. Fast Directional Gradient Descent Search (FDGDS) Search centre 1st stage 2nd stage 3rd stage Best matchedL.-M. Po, K.-H. Ng, K.-W. Cheung, K.-M. Wong, Y. Uddin, and C.-W. Ting, “Novel Directional Gradient Descent Searches for Fast Block MotionEstimation,” IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 8, 2009 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 36
  37. 37. Quality MetricsPeak Signal-to-Noise Ratio Structural Similarity Index It is a point to point metric  It is a windowed metric Based on square differences  Based on luminance, contrast and It is not very well matched to perceived structure between an original and a visual quality distorted images  It takes into account the visual perception of the imageZ. Wang, A. C. Bovik, H. R. Sheikh and, E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans.Image Process., vol. 13, no. 4, 2004C. S. varnan, A. Jagan, J. Kaur, D. Jyoti, and D. S. Rao, “Image quality assessment techniques pn spatial domain,” International Journal onComputer Science and Technology, vol. 2, no. 3, 2011 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 37
  38. 38. Quality Metrics (ii) PSNR = 26.547, SSIM = 0.988 PSNR = 26.547, SSIM = 0.913 Original image PSNR undefinied, SSIM = 1 PSNR = 26.547, SSIM = 0.840 PSNR = 26.547, SSIM = 0.694https://ece.uwaterloo.ca/~z70wang/research/ssim/ A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 38
  39. 39. Performance Metrics  Since an algorithm requires time proportional to the number of explored blocks (EXB), the computational cost of a BMA is determined by the EXB EXB in the case of Zero Motion Vector (ZMV) BMA 3SS 4SS DS HEXBS MDGDS FDGDS EXB 25 17 13 11 9 9V. Padilla, “Algoritmos de block-matching para compresión de video,” Final Career Project, Systems Engineering Program, Universidad delValle, 2009 A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 39
  40. 40. Video Test SequencesSequence Size # Frames MotionAkiyo 352x288 300 SmallMother_ 352x288 300 SmalldaugthterSilent 352x288 300 SmallForeman 352x288 300 MediumGarden 352x240 115 MediumMobile 352x288 300 MediumCoastguard 352x288 300 LargeFootball 352x288 260 LargeStefan 352x240 300 LargeBlock sizes used: 8x8, 16x16 and 32x32 All video sequences used are in uncompressed format: YUV4MPEG, and are available at: http://media.xiph.org/video/derf/ A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 40
  41. 41. Results PSNR performance, block size of 8x8 pixels 27,000 FS MDGDS FDGDS DS 26,000 4SS 3SS MDGDS FS FDGDS FS 25,000 4SS HEXBSPSNR (dB) 24,000 3SS MDGDS FDGDS DS 4SS 23,000 HEXBS 3SS HEXBS 22,000 DS 21,000 20,000 Football Garden Stefan Video Sequence A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 41
  42. 42. Results (ii) SSIM performance, block size of 8x8 pixels MDGDS 0,950 FDGDS FS 4SS 0,900 HEXBS FS 3SS 0,850 MDGDS FDGDS MDGDS DS 3SS FDGDS 4SS DS 0,800 FS DS 4SS 3SSSSIM HEXBS 0,750 0,700 HEXBS 0,650 0,600 Football Garden Stefan Video Sequence A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 42
  43. 43. Results (iii) EXB performance, block size of 8x8 pixels 32,000 MDGDS 27,000 FDGDS 4SS 3SS 3SS 3SS 3SS 4SS 22,000 4SS DS MDGDSEXB MDGDS FDGDS 4SS DS 17,000 DS FDGDS DS HEXBS HEXBS HEXBS HEXBS 12,000 MDGDS FDGDS 7,000 Akiyo Football Garden Stefan Video Sequence A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 43
  44. 44. Results (iv) SSIM/EXB performance, block size 8x8 pixels MDGDS 0,120 FDGDS 0,100 HEXBS HEXBS DS FDGDS HEXBS 0,080 HEXBS MDGDSSSIM / EXB 4SS FDGDS 0,060 MDGDS DS DS 4SS 3SS 4SS DS FDGDS 3SS MDGDS 3SS 0,040 3SS 4SS 0,020 0,000 Akiyo Football Garden Stefan Video Sequence A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 44
  45. 45. Results (v) SSIM performance of various algorithms for Coastguard video sequence 0,94 0,92 0,9 0,88SSIM 3SS 4SS 0,86 DS HEXBS 0,84 0,82 0,8 8x8 16x16 32x32 Block Size A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 45
  46. 46. Results (vi) SSIM performance of various algorithms for Football video sequence 0,8 0,75 0,7SSIM 0,65 3SS 4SS DS 0,6 HEXBS 0,55 0,5 8x8 16x16 32x32 Block Size A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 46
  47. 47. Results (vii) SSIM performance of various algoritms for Garden video sequence 0,9 0,88 0,86 0,84 0,82SSIM 0,8 3SS 4SS 0,78 DS HEXBS 0,76 0,74 0,72 0,7 8x8 16x16 32x32 Block Size A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 47
  48. 48. Final Remarks The HEXBS shows low computational cost but produces low quality of prediction The MDGDS and the FDGDS show low computational cost and produce the highest quality of prediction The FGDGS achieves a good trade off between high quality of prediction and a low computation cost The HEXBS is less affected by the variation in the block sizes, whilst the others show a big loss of prediction by increasing the block size used A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 48
  49. 49. A Comparison of Block-Matching Motion Estimation Algorithms, 7CCC 2012, Medellín - Colombia Slide 49

×