Aruna Ravi - M.S Thesis

3,571 views

Published on

MS-EE Thesis defense presentation on Dirac video codec

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,571
On SlideShare
0
From Embeds
0
Number of Embeds
27
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Aruna Ravi - M.S Thesis

  1. 1. - Aruna Ravi MS-EE Student , UTA [email_address] July 17 th 2009
  2. 2. Overview <ul><li>Dirac - overview </li></ul><ul><li>H.264/MPEG-4 AVC - overview </li></ul><ul><li>Dirac performance analysis based on quality factor </li></ul><ul><li>Performance comparison of Dirac with H.264 at constant bit rates </li></ul><ul><li>Results </li></ul><ul><li>Conclusions </li></ul><ul><li>Future research </li></ul><ul><li>References </li></ul>
  3. 3. What is Dirac? <ul><li>Motion-compensated state-of-the-art hybrid video codec developed by BBC. </li></ul><ul><li>Uses modern techniques - wavelet transforms, arithmetic coding. </li></ul><ul><li>Open technology – no license fees. </li></ul><ul><li>Easily recreated for new platforms. </li></ul><ul><li>Applications - HDTV to web streaming. </li></ul>
  4. 4. Dirac encoder / decoder blocks <ul><li>Transform: 2D discrete wavelet transform </li></ul><ul><li>Scaling and Quantization: Dead zone quantization </li></ul><ul><li>Motion estimation: Hierarchical motion estimation </li></ul><ul><li>Motion compensation: Overlapped block-based motion compensation (OBMC) </li></ul><ul><li>Entropy coding: Arithmetic coding </li></ul>
  5. 5. Dirac - Architecture <ul><li>V in : Input video frame; P : Previous frame; Motion compensated prediction error (MCPE): e = V in – P ; e TQ : MCPE after wavelet transformation and quantization; e’ : MCPE after inverse transformation; V local = P + e’ </li></ul>
  6. 6. Dirac encoder / decoder blocks <ul><li>Transform: 2D discrete wavelet transform </li></ul><ul><li>Scaling and Quantization: Dead zone quantization </li></ul><ul><li>Motion estimation: Hierarchical motion estimation </li></ul><ul><li>Motion compensation: Overlapped block-based motion compensation (OBMC) </li></ul><ul><li>Entropy coding: Arithmetic coding </li></ul>
  7. 7. Dirac - Wavelet Transform <ul><li>More efficient than block transforms with still images. </li></ul><ul><li>Operates on entire picture. </li></ul><ul><li>Flexible – operates at a range of resolutions. </li></ul><ul><li>Repeated filtering of signals into low- and high-frequency parts. </li></ul><ul><li>Horizontal and vertical filtering for 2D signals. </li></ul><ul><li>Logarithmic frequency decomposition into sub-bands at each stage. </li></ul><ul><li>Daubechies wavelet filters </li></ul>
  8. 8. Wavelet Transform contd.. <ul><li>Packs most information into low frequency sub-bands - compression achieved. </li></ul><ul><li>Process repeated to achieve higher levels of wavelet transform. </li></ul><ul><li>Coded picture free from block artifacts, superior moving images. </li></ul>
  9. 9. Stages of wavelet transform
  10. 10. Dirac encoder / decoder blocks <ul><li>Transform: 2D discrete wavelet transform </li></ul><ul><li>Scaling and Quantization: Dead zone quantization </li></ul><ul><li>Motion estimation: Hierarchical motion estimation </li></ul><ul><li>Motion compensation: Overlapped block-based motion compensation (OBMC) </li></ul><ul><li>Entropy coding: Arithmetic coding </li></ul>
  11. 11. Dirac - Scaling and Quantization <ul><li>Scaling - Extracting frame data after wavelet transform, scaling the coefficients to perform quantization. </li></ul><ul><li>Quantization - rate distortion optimization, strip information from frame data that results in as little visual distortion as possible. </li></ul><ul><li>Dead-zone quantization - coarser quantization on smaller values. </li></ul>
  12. 12. Dirac encoder / decoder blocks <ul><li>Transform: 2D discrete wavelet transform </li></ul><ul><li>Scaling and Quantization: Dead zone quantization </li></ul><ul><li>Motion estimation: Hierarchical motion estimation </li></ul><ul><li>Motion compensation: Overlapped block-based motion compensation (OBMC) </li></ul><ul><li>Entropy coding: Arithmetic coding </li></ul>
  13. 13. Dirac - Motion Estimation <ul><li>Exploits temporal redundancy in video streams. </li></ul><ul><li>Hierarchical motion estimation - three stages. </li></ul><ul><li>Stage I: Pixel accurate MV determined for each block, reference frame by hierarchical block matching. </li></ul><ul><li>Stage II: Pixel-accurate vectors refined. </li></ul><ul><li>Stage III: Mode decisions made for each macro-block. </li></ul>
  14. 14. Intra & Inter prediction <ul><li>Three types of frames : I (Intra), L1 (Level 1) and L2 (Level 2). </li></ul><ul><li>I frames coded without reference. </li></ul><ul><li>Inter frames - L1, L2 frames coded with reference to previously coded frames. </li></ul><ul><li>L1 frames used as temporal references for other frames, L2 frames are not. </li></ul><ul><li>Each frame predicted from up to two reference frames. </li></ul><ul><li>Prediction modes: Intra, Reference 1 only, Reference 2 only, Bi-directional. </li></ul>
  15. 15. Dirac encoder / decoder blocks <ul><li>Transform: 2D discrete wavelet transform </li></ul><ul><li>Scaling and Quantization: Dead zone quantization </li></ul><ul><li>Motion estimation: Hierarchical motion estimation </li></ul><ul><li>Motion compensation: Overlapped block-based motion compensation (OBMC) </li></ul><ul><li>Entropy coding: Arithmetic coding </li></ul>
  16. 16. Dirac - Motion Compensation <ul><li>Overlapped block-based motion compensation - good compression, prevents blocky edges. </li></ul><ul><li>Interaction of neighboring blocks. </li></ul><ul><li>Basic blocks arranged into macro-blocks (4x4). </li></ul><ul><li>Data padding for exact number of horizontal and vertical macro-blocks. </li></ul><ul><li>Smaller blocks selected for lower resolution frames. </li></ul><ul><li>Linear ramp mask - weight function on predicting block. </li></ul>
  17. 17. Motion Compensation contd.. <ul><li>Splitting of macro-blocks </li></ul><ul><li>Default (streaming, SD) -12x12 blocks, overlapped at intervals of 8 pixels vertically and horizontally. </li></ul><ul><li>Sub-pixel MC with MV - 1/8th pixel accuracy. </li></ul><ul><li>MV predicts current frame identical to previous frame at low bit rates, minimizes cost of encoding residual data. </li></ul>
  18. 18. Dirac encoder / decoder blocks <ul><li>Transform: 2D discrete wavelet transform </li></ul><ul><li>Scaling and Quantization: Dead zone quantization </li></ul><ul><li>Motion estimation: Hierarchical motion estimation </li></ul><ul><li>Motion compensation: Overlapped block-based motion compensation (OBMC) </li></ul><ul><li>Entropy coding: Arithmetic coding </li></ul>
  19. 19. Dirac - Entropy Coding <ul><li>Three stages: binarization, context modeling and arithmetic coding. </li></ul><ul><li>First stage provides bit stream with easily analyzable statistics, encoded using arithmetic coding. </li></ul><ul><li>Context modeling - coefficient is small or not, well-predicted by its neighbors and parents. </li></ul><ul><li>Arithmetic coding performs lossless compression - flexible and efficient. </li></ul>
  20. 20. Advantages of Dirac <ul><li>Open technology - no licensing costs. </li></ul><ul><li>Flexible, ease of operation over many applications. </li></ul><ul><li>Capable of compressing high resolution files. </li></ul><ul><li>Exploits lossy compression using long group of picture (GOP) formats for broadcasting and other distribution outlets. </li></ul><ul><li>Quality similar to that of AVC/H.264. </li></ul><ul><li>More freedom for future implementations, more scope to improve performance </li></ul><ul><li>Comparable to other codecs despite simple toolset. </li></ul><ul><li>Relative simplicity and clean architecture supports high performance. </li></ul><ul><li>Good quality at low bit rates, lower costs. </li></ul>
  21. 21. Applications of Dirac <ul><li>Broadcasting </li></ul><ul><li>Clip distribution </li></ul><ul><li>Live streaming video </li></ul><ul><li>Pod casting </li></ul><ul><li>Creative archive </li></ul><ul><li>Peer to peer transfers </li></ul><ul><li>HDTV with SD (standard definition) simulcast capability </li></ul><ul><li>Higher density channel packing </li></ul><ul><li>Desktop production </li></ul><ul><li>News links </li></ul><ul><li>Archive storage </li></ul><ul><li>PVRs (personal video recorder) </li></ul><ul><li>Multilevel Mezzanine coding (aka SMPTE VC-2) </li></ul>
  22. 22. Overview of H.264 / AVC <ul><li>Open, licensed standard, latest block-oriented motion-compensation-based codec. </li></ul><ul><li>Good video quality at substantially lower bit rates. </li></ul><ul><li>Better rate-distortion performance and compression efficiency without increasing design complexity. </li></ul><ul><li>Simple syntax specifications, very flexible. </li></ul><ul><li>Wide variety of applications such as video broadcasting, video streaming, video conferencing, D-Cinema, HDTV. </li></ul><ul><li>Network friendly; demonstrates a balance between coding efficiency, implementation complexity and cost. </li></ul><ul><li>Layered structure - consists of two layers: Network Abstraction Layer (NAL) and Video Coding Layer (VCL); supports 4:2:0 chroma sampling picture format including QCIF and CIF formats. </li></ul>
  23. 23. Overview of H.264 contd.. <ul><li>Uses hybrid block based video compression techniques such as: </li></ul><ul><ul><li>Transformation - reduction of spatial correlation </li></ul></ul><ul><ul><li>Quantization - bit-rate control </li></ul></ul><ul><ul><li>Motion compensated prediction - reduction of temporal correlation </li></ul></ul><ul><ul><li>Entropy coding - reduction in statistical correlation </li></ul></ul><ul><li>Includes the following features: </li></ul><ul><ul><li>Intra-picture prediction </li></ul></ul><ul><ul><li>4x4 integer transform </li></ul></ul><ul><ul><li>Multiple reference pictures </li></ul></ul><ul><ul><li>Variable block sizes </li></ul></ul><ul><ul><li>Quarter pel precision for motion compensation </li></ul></ul><ul><ul><li>In-loop de-blocking filter </li></ul></ul><ul><ul><li>Improved entropy coding </li></ul></ul>
  24. 24. H.264 / MPEG-4 AVC architecture
  25. 25. Algorithmic Element Comparison
  26. 26. Quality factor in Dirac <ul><li>Overall trade-off factor derived from value termed “QF” - quality/quantization factor. </li></ul><ul><li>Picture lambda values used for rate-distortion control of quantization and motion estimation </li></ul><ul><li>Initially derived from picture QF: </li></ul><ul><ul><li>Set directly on command line, used for all pictures </li></ul></ul><ul><ul><li>Determined by means of Rate Control algorithm </li></ul></ul>
  27. 27. Dirac Performance Analysis based on Quality Factor
  28. 28. Dirac Performance Analysis based on QF contd..
  29. 29. Dirac Performance Analysis based on QF contd..
  30. 30. Dirac Performance Analysis based on QF contd..
  31. 31. Dirac Performance Analysis based on QF contd..
  32. 32. Dirac Performance Analysis based on QF contd..
  33. 33. Dirac performance analysis test results
  34. 34. Dirac performance analysis test results contd..
  35. 35. Dirac performance analysis test results contd.. Point of lossless coding Good SSIM even at lower QF for CIF !
  36. 36. Dirac performance analysis test results contd..
  37. 37. Dirac performance analysis test results contd..
  38. 38. Performance comparison of Dirac with H.264 at constant bit rates (CBR) <ul><li>Test sequence: “ Miss-america.yuv ” , QCIF (150 frames) </li></ul>Dirac performs better than H.264 at lower bit rates!
  39. 39. Performance comparison of Dirac with H.264 at constant bit rates (CBR) contd.. Results comparable to H.264
  40. 40. Performance comparison of Dirac with H.264 at constant bit rates (CBR) contd.. <ul><li>Test sequence: “ Stefan.yuv ” , CIF (90 frames) </li></ul>
  41. 41. Performance comparison of Dirac with H.264 at constant bit rates (CBR) contd..
  42. 42. Performance comparison of Dirac with H.264 at constant bit rates (CBR) contd.. <ul><li>Test sequence: “ Susie.yuv ” , SDTV (25 frames) </li></ul>
  43. 43. Performance comparison of Dirac with H.264 at constant bit rates (CBR) contd..
  44. 44. Performance comparison of Dirac with H.264 at constant bit rates (CBR) contd.. Almost similar compression achieved !
  45. 45. Performance comparison of Dirac with H.264 at constant bit rates (CBR) contd.. Note the scale of difference!
  46. 46. Performance comparison of Dirac with H.264 at constant bit rates (CBR) contd.. Consistent performance over large range of bit rates
  47. 47. Performance comparison of Dirac with H.264 at constant bit rates (CBR) contd..
  48. 48. Conclusions <ul><li>Dirac is very comparable to H.264. </li></ul><ul><li>Dirac achieves slightly higher compression ratios for lower bitrates than H.264 in case of QCIF sequences. </li></ul><ul><li>At higher QCIF bitrates both Dirac and H.264 achieve similar compression. </li></ul><ul><li>H.264 provides slightly better compression especially for CIF, SD media. </li></ul><ul><li>H.264 has slightly better SSIM than Dirac for all the test sequences. </li></ul><ul><li>H.264 achieves better PSNR and lower MSE than Dirac. </li></ul><ul><li>Dirac’s encoder is less complex and fast encoding (about 5-10 minutes) is possible. H.264 takes a longer time (around 25 minutes) to encode the same video sequence. </li></ul><ul><li>Decoder complexity of Dirac and H.264 is nearly the same. Both codecs require about the same time to perform decoding. </li></ul><ul><li>Dirac is simple yet robust codec, nearly equaling H.264 in performance, sans the cost of royalty fees. </li></ul>
  49. 49. Future Research <ul><li>Dirac video codec can be further improved by integrating it with audio codecs such as AAC/ MP2/ Vorbis/ FLAC. </li></ul><ul><li>According to Dr. T. Borer, Dirac team leader, BBC: </li></ul><ul><ul><li>Dirac wrapped in container format can be used with audio </li></ul></ul><ul><ul><li>Put into transport streams with MPEG-TS supported audio format </li></ul></ul><ul><li>Transcoding possible by: </li></ul><ul><ul><li>Multiplexing video and audio coded bit streams to create a single bit stream for transmission and de-multiplexing streams at receiving end. </li></ul></ul><ul><ul><li>Synchronization of audio and video during playback. </li></ul></ul><ul><li>DiracPRO / SMPTE VC-2: </li></ul><ul><ul><li>Intra frame version of Dirac </li></ul></ul><ul><ul><li>Non proprietary, royalty free, open technology </li></ul></ul><ul><ul><li>Intended for high quality applications with lower compression ratios </li></ul></ul><ul><ul><li>Optimized for professional production , archiving applications </li></ul></ul><ul><ul><li>More significant transcoding long GOP from H.264 </li></ul></ul>
  50. 50. REFERENCES <ul><li>[1] T. Borer, and T. Davies, “Dirac video compression using open technology”, BBC EBU Technical Review, July 2005 </li></ul><ul><li>[2] BBC Research on Dirac: http:// www.bbc.co.uk/rd/projects/dirac/index.shtml </li></ul><ul><li>[3] H. Eeckhaut, et al, “Speeding up Dirac’s entropy coder”, Proc. 5th WSEAS Int. Conf. on Multimedia, Internet and Video Technologies, pp. 120-125, Greece, Aug. 2005. </li></ul><ul><li>[4] The Dirac web page: http:// dirac.sourceforge.net </li></ul><ul><li>[5] K. Onthriar, K. K. Loo and Z. Xue, “Performance comparison of emerging Dirac video codec with H.264/AVC”, IEEE International Conference on Digital Telecommunications, 2006, ICDT; Vol. 06, Page: 22, Issue: 29-31, Aug. 2006. </li></ul><ul><li>[6] T.Davies , “The Dirac Algorithm” http:// dirac.sourceforge.net /documentation/algorithm/ , 2005. </li></ul><ul><li>[7] M. Tun and W. A. C. Fernando, “An error-resilient algorithm based on partitioning of the wavelet transform coefficients for a DIRAC video codec” , Tenth International Conference on Information Visualization, 2006, IV, Vol. 5-7 , pp.: 615 – 620, Issue : July 2006. </li></ul><ul><li>[8] T. Davies, “A modified rate-distortion optimization strategy for hybrid wavelet video coding” , IEEE International Conference on Acoustics, Speech and Signal Processing, 2006. ICASSP 2006 Proceedings. 2006, Vol.: 2, pp.: II, Publication Date: 14-19 May 2006. </li></ul><ul><li>[9] M. Tun, K. K. Loo and J. Cosmas, “Semi-hierarchical motion estimation for the Dirac video codec”, 2008 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting, pp.: 1-6, Publication Date: March 31 2008-April 2 2008. </li></ul><ul><li>[10] /CMPT 365 Course Slides/, School of Computing Science, Simon Fraser University, fig3: http://www.cs.sfu.ca/CourseCentral/365/li/material/notes/Chap4/Chap4.3/Chap4.3.html </li></ul><ul><li>[11] Dirac developer support: Overlapped block-based motion compensation : http://dirac.sourceforge.net/documentation/algorithm/algorithm/toc.htm </li></ul><ul><li>[12] S.K.Kwon, A.Tamhankar and K.R.Rao, “Overview of H.264 / MPEG-4 Part 10” J. Visual Communication and Image Representation, Vol 17, pp.186-216, April 2006. </li></ul><ul><li>[13] T.Wiegand, et al “Overview of the H.264/AVC video coding standard”, IEEE Trans. CSVT, Vol.13, pp 560-576, July 2003. </li></ul>
  51. 51. REFERENCES contd.. <ul><li>[14] T. Wiegand and G. J. Sullivan, “The H.264 video coding standard”, IEEE Signal Processing Magazine, vol. 24, pp. 148-153, March 2007. </li></ul><ul><li>[15] D. Marpe, T. Wiegand and G. J. Sullivan, “The H.264/MPEG-4 AVC standard and its applications”, IEEE Communications Magazine, vol. 44, pp. 134-143, Aug. 2006. </li></ul><ul><li>[16] I. Richardson, “The H.264 AVC standard”, to be published in late 2009. </li></ul><ul><li>[17]  C. Gargour et al, “A short introduction to wavelets and their applications”, IEEE Circuits and Systems Magazine, vol. 9, pp. 57-68, II Quarter, 2009. </li></ul><ul><li>[18] “Dirac Pro to bolster BBC HD links”: </li></ul><ul><li>http://www.broadcastnow.co.uk/news/multi-platform/news/dirac-pro-to-bolster-bbc-hd-links/1732462.article </li></ul><ul><li>[19] “And now, Dirac from the Olympics, a new free codec!”: http://www.videsignline.com/210601739 </li></ul><ul><li>[20] ‘Dirac codec’: http:// en.wikipedia.org/wiki/Dirac_(codec ) </li></ul><ul><li>[21] ‘Video encoding’: http:// www.webopedia.com/TERM/V/video_encoding.html </li></ul><ul><li>[22] “Video encoding and bandwidth requirements”: http:// www.cybertechmedia.com/videoencoding.html </li></ul><ul><li>[23] Z. Wang, et al “Image quality assessment: From error visibility to structural similarity”, IEEE Trans. on Image Processing, vol. 13, pp. 600-612, Apr. 2004. </li></ul><ul><li>http://www.ece.uwaterloo.ca/~z70wang/ </li></ul><ul><li>[24] http:// en.wikipedia.org/wiki/Windows_Media_Video </li></ul><ul><li>[25] Microsoft Windows Media: http://www.microsoft.com/windows/ windowsmedia </li></ul><ul><li>[26] MPEG-4 Part 2, ISO/IEC 14496-2, International Organization for Standardization, http://www.iso.ch </li></ul><ul><li>[27] http://en.wikipedia.org/wiki/MPEG-4_Part_2 </li></ul><ul><li>[28] Dirac software and source code: http://diracvideo.org/download/dirac-research/ </li></ul><ul><li>[29] http://en.wikipedia.org/wiki/VC-1 </li></ul>
  52. 52. REFERENCES contd.. <ul><li>[30] Dirac video codec - A programmer's guide: </li></ul><ul><li>http://dirac.sourceforge.net/documentation/code/programmers_guide/toc.htm </li></ul><ul><li>[31] H. Jia and L. Zhang, “Directional diamond search pattern for fast block motion estimation”, IEE Electronics Letters, Vol. 39, No. 22, pp. 1581-1583, 30th October 2003. </li></ul><ul><li>[32] Video test sequences (YUV 4:2:0): http:// trace.eas.asu.edu/yuv/index.html </li></ul><ul><li>[33] Video test sequences ITU601: http://www.cipr.rpi.edu/resource/sequences/itu601.html </li></ul><ul><li>[34] MPEG–2 advanced audio coding, AAC. International Standard IS 13818–7, ISO/IEC JTC1/SC29 WG11, 1997 </li></ul><ul><li>[35] G. A. Davidson, et al “ATSC video and audio coding”, Proceedings of IEEE, vol. 94, pp. 60-76, Jan. 2006 </li></ul><ul><li>[36] A. Puri, X. Chen and A. Luthra, “Video coding using the H.264/MPEG-4 AVC compression standard”, Signal Processing: Image Communication Magazine, vol. 19, pp. 793-849, Oct. 2004 </li></ul><ul><li>[37] H.264 AVC JM software: http://iphome.hhi.de/suehring/tml/ </li></ul><ul><li>[38] SSIM: http://en.wikipedia.org/wiki/SSIM </li></ul><ul><li>[39] Daubechies wavelet: http://en.wikipedia.org/wiki/Daubechies_wavelet </li></ul><ul><li>[40] Daubechies wavelet filter design: http://cnx.org/content/m11159/latest/ </li></ul><ul><li>[41] Dirac developer support: Wavelet transform: http://dirac.sourceforge.net/documentation/algorithm/algorithm/wlt_transform.xht </li></ul><ul><li>[42] Dirac developer support: RDO motion estimation metric: </li></ul><ul><li>http://dirac.sourceforge.net/documentation/algorithm/algorithm/rdo_mot_est.xht </li></ul><ul><li>[43] Digital down converter: http:// en.wikipedia.org/wiki/Digital_down_converter </li></ul><ul><li>[44] T. Jacob, K. R. Rao and D. N. Kim, “Image Mirroring, Rotation and Interpolation in Wavelet Domain”, Applications of Digital Image Processing XXXII conference, SPIE symposium on Optical Engineering, 2-6 Aug. 2009. </li></ul><ul><li>[45] H.264/MPEG-4 AVC: http://en.wikipedia.org/wiki/H.264 </li></ul>
  53. 53. REFERENCES contd.. <ul><li>[46] M.Fieldler, “Implementation of basic H.264/AVC Decoder”, seminar paper at Chemnitz University of Technology, June 2004. </li></ul><ul><li>[47] H.264 encoder and decoder: http://www.adalta.it/Pages/407/266881_266881.jpg </li></ul><ul><li>[48] “H.264 video compression standard”, White paper, Axis communications. </li></ul><ul><li>[49] MPEG-4: ISO/IEC JTC1/SC29 14496-10: Information technology – Coding of audio-visual objects - Part 10: Advanced Video Coding, ISO/IEC, 2005. </li></ul><ul><li>[50] D. Kumar, P. Shastry and A. Basu, “Overview of the H.264 / AVC”, 8th Texas Instruments Developer Conference India, 30 Nov – 1 Dec 2005, Bangalore. </li></ul><ul><li>[51] R. Schäfer, T. Wiegand and H. Schwarz, “The emerging H.264/AVC standard”, EBU Technical Review, Jan. 2003. </li></ul><ul><li>[52] Joint Photographic Experts Group, JPEG: http:// www.jpeg.org / </li></ul><ul><li>[53] FLAC - Free Lossless Audio Codec: http:// flac.sourceforge.net / </li></ul><ul><li>[54] Vorbis: http:// www.vorbis.com / </li></ul><ul><li>[55] MPEG Layer II: http://en.wikipedia.org/wiki/MPEG-1_Audio_Layer_II </li></ul><ul><li>[56] MPEG-TS / MPEG transport stream: http:// en.wikipedia.org/wiki/MPEG_transport_stream </li></ul><ul><li>[57] MP3 / MPEG Layer III: http://en.wikipedia.org/wiki/MP3 </li></ul><ul><li>[58] Dirac Pro: http:// www.bbc.co.uk/rd/projects/dirac/diracpro.shtml </li></ul><ul><li>[59] T. Borer, “Dirac coding: Tutorial & Implementation”, EBU Networked Media Exchange seminar, June 2009. </li></ul><ul><li>[60] DiracPRO software: http:// dirac.kw.bbc.co.uk /download/ </li></ul>
  54. 56. Performance metrics <ul><li>Compression ratio quantifies reduction in data size due to data compression. </li></ul><ul><li>Bitrate refers to the number of bits used per unit of playback time to represent a continuous medium such as video after data compression. </li></ul>
  55. 57. Performance metrics contd.. <ul><li>Mean squared error (MSE) for two m × n monochrome images I and K (one considered a noisy approximation of the other) is given by: </li></ul><ul><li>Peak signal-to-noise ratio (PSNR): Ratio between maximum possible power of a signal and power of corrupting noise that affects the fidelity of its representation. Expressed in terms of the logarithmic decibel scale. </li></ul><ul><li>where MAX I is the maximum possible pixel value of the image </li></ul>
  56. 58. Performance metrics contd.. <ul><li>Structural Similarity (SSIM) index is a method for measuring similarity between two images. </li></ul><ul><li>Compares local patterns of pixel intensities normalized for luminance and contrast. </li></ul><ul><li>Full reference metric, measurement of image quality based on original uncompressed image as reference. </li></ul><ul><li>SSIM between two windows of size N x N x and y is given by: </li></ul>
  57. 59. H.264 profiles
  58. 60. H.264 encoder / decoder architecture
  59. 61. H.264 - Transform <ul><li>4x4 multiplier-free integer transform </li></ul><ul><li>Transform coefficients perfectly invertible </li></ul><ul><li>Hierarchical structure - 4 x 4 Integer DCT, Hadamard transform </li></ul><ul><li>Hadamard transform applied when (16x16) intra prediction mode is used with (4x4) integer DCT </li></ul><ul><li>MB size for chroma depends on 4:2:0, 4:2:2 and 4:4:4 formats </li></ul>
  60. 62. H.264 - Scaling and Quantization <ul><li>Multiplication operation for exact transform combined with multiplication of scalar quantization </li></ul><ul><li>Scale factor for each element in each sub-block varies as a function of quantization parameter associated with macro-block that contains sub block; position of element within sub-block </li></ul><ul><li>Rate-control algorithm in encoder controls value of quantization parameter </li></ul><ul><li>Encoder performs post-scaling and quantization </li></ul>
  61. 63. H.264 – Entropy Coding <ul><li>CAVLC (Context-based Adaptive Variable Length Coding). </li></ul><ul><li>CABAC (Context-based Adaptive Binary Arithmetic Coding). </li></ul><ul><li>CAVLC makes use of run-length encoding. </li></ul><ul><li>CABAC utilizes arithmetic coding; codes both MV and residual transform coefficients. </li></ul><ul><li>Typically CABAC provides 10-15 % reduction in bit rate compared to CAVLC, for the same PSNR. </li></ul><ul><li>All syntax elements are encoded by Exp-Golomb codes (Universal Variable Length Codes (UVLC)). </li></ul>
  62. 64. H.264 – Deblock filtering <ul><li>In-Loop deblock filter – smoothes blocky edges; increases rate-distortion performance. </li></ul><ul><li>Frame decode process - top/left neighbors fully reconstructed and available as inputs for de-blocking current MB. </li></ul><ul><li>Applied to all 4x4 blocks except at picture boundaries. </li></ul><ul><li>Vertical edges filtered first (left to right) followed by horizontal edges (top to bottom). </li></ul>
  63. 65. H.264 – Intra Prediction <ul><li>Spatial domain prediction </li></ul><ul><li>Uses reconstructed data from neighboring macro-blocks to predict current macro-block coefficients. </li></ul><ul><li>Prediction from samples in pixel domain; flexible bit-rate control; eliminates redundancies across multiple directions. </li></ul>
  64. 66. Intra Prediction: Mode decisions for 4x4 blocks <ul><li>For Luminance pixels: </li></ul><ul><ul><li>4x4 partition: 9 intra-prediction modes </li></ul></ul><ul><ul><li>16x16 partition: 4 intra-prediction modes </li></ul></ul><ul><li>16 x 16 mode - better compression for flat regions of picture, lower costs. </li></ul><ul><li>Avoids gradients in flat regions with high quantization parameters </li></ul>
  65. 67. H.264 – Inter Prediction <ul><li>Motion estimation, Motion compensation included. </li></ul><ul><li>Generates predicted version of a rectangular array of pixels. </li></ul><ul><li>Similar sized rectangular arrays of pixels chosen from previously decoded reference pictures. </li></ul><ul><li>Reference arrays translated to current rectangular array positions. </li></ul><ul><li>Half-pel interpolated using a 6-tap FIR filter with weights (1, -5, 20, 20, -5, 1)/32. </li></ul><ul><li>Quarter-pel produced using bilinear interpolation between neighboring half- or integer-pel samples. </li></ul>
  66. 68. H.264 – Motion Compensation <ul><li>Predicted rectangular arrays of pixels: 4x4, 4x8, 8x4, 8x8, 16x8, 8x16, and 16x16. </li></ul><ul><li>Translation from other array positions in reference picture specified with quarter pixel precision. </li></ul><ul><li>In case of 4:2:0 format, the chroma MVs have a resolution of 1/8 of a pixel, derived from transmitted luma MVs of 1/4 pixel resolution. </li></ul>
  67. 69. H.264 – Multi-Picture Motion Compensation <ul><li>More than one prior-coded picture used as a reference for motion-compensated prediction. </li></ul><ul><li>Encoder, decoder store reference pictures in a multi-picture buffer. </li></ul><ul><li>Decoder replicates multi-picture buffer of the encoder, according to reference picture buffering type specified in the bit stream. </li></ul>

×