Your SlideShare is downloading. ×
Introduction To Video Compression
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Introduction To Video Compression

4,511
views

Published on

Introduction to model based compression used for multimedia and spefically video compression

Introduction to model based compression used for multimedia and spefically video compression

Published in: Technology

0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,511
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
412
Comments
0
Likes
5
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Introduction to Video Compression Fast Forward Your Development www.dsp-ip.com
  • 2. Compression p • Compression – Representing information by less bit than the original information • Lossless Compression – Original information and compressed information are identical identical. example LZ, TAR and other compression techniques. techniques • Lossy Compression – Compressed info is not the same as uncompressed info E ample MP3 ncompressed info. Example: MP3, JPEG etc •LLossy compression is often MODEL B i i ft Based d Compression Fast Forward Your Development www.dsp-ip.com
  • 3. Compression terms p • Encoder – Module which compress the information • Decoder – Module which decompress the information • CODEC – (en)CODer / DEcoder • Channel – the medium which the information is passed through for example ADSL line or disk Channel Decoder Encoder Disk Fast Forward Your Development www.dsp-ip.com
  • 4. Sampling p g • Digitization of analog signal • Voice - Sampling in time only The number of only. samples per second is the Frequency – F. in telephony, telephony voice sampling frequency is 8KHz to transfer 4KHz of sound. •IImage - SSampling in 2D space - F F O thi li i Fx, Fy. On this screen the images are sampled 1024x768 • Vid - S Video Sampling i 2D space and ti li in D d time – F F Fx, Fy, Fn. On TV Fn is 25 or 30 fps (frames per second) Fast Forward Your Development www.dsp-ip.com
  • 5. Quantization • Lowering the number of bits needed to describe an object object. • For example I scored 66 in the test an object with 100 steps need 7 description bits bits. • How is she looking ? 10 ! Needs only 5 bits for description. d i ti • Could we lower the number of bits more? What is th i the minimum? i i ? • Quantization is not always linear - ADPCM Fast Forward Your Development www.dsp-ip.com
  • 6. MODEL BASED COMPRESSION Voice i Audio Video Fast Forward Your Development www.dsp-ip.com
  • 7. Model Base Transform - example • Transforming the information from a representation of similar importance to all variables to a representation of varying importance • Th “poor sheriff” problem The “ h iff” bl ▫ The sheriff needs to telegram the dangers criminal profile t d i i l fil to headquarters. ▫ A telegram costs a cent for each bit. ▫ Sending it bit by bit would finish the poor sheriff salary. Fast Forward Your Development www.dsp-ip.com
  • 8. Solution • The sheriff and headquarters agree on a b base that would h ld describe images according to a base known to both sides. • The transformation base is: ▫ Mouth ▫ Nose ▫ Eyes ▫ Ears • The values are the sizes of each item quantized to millimeters • Mad Max will be transmitted as: 8,14,12,17 8 14 12 17 Fast Forward Your Development www.dsp-ip.com
  • 9. Model Based Compression (Multimedia) Pre- Model Based Quantize / Entropy Processing Transform Prioritize Reorder Codingg Fast Forward Your Development www.dsp-ip.com
  • 10. Human Visual System • The human eye has two basic light receptors: ▫ Rods – Light Intensity receptors ▫ Cons – Colored light receptors • Fast Forward Your Development www.dsp-ip.com
  • 11. The Human Eye • Rods Concentration >> Cons Concentration • Green Discrimination << Red, Blue Red Discrimination •LLow Frequency > High Frequency F Hi h F Fast Forward Your Development www.dsp-ip.com
  • 12. Model Based transformations • RGB (3 equally quantized colors) -> > • YUV (Light Intensity + two color channels) • Pi l based d Pixel b d domain -> F i > Frequency d domain i Fast Forward Your Development www.dsp-ip.com
  • 13. Speech coding • In speech coding, the vocal tract is used as a model: Fast Forward Your Development www.dsp-ip.com
  • 14. Audio / Music Coding • In general Audio Coding, the ear is used as a model: • Frequencies -> Frequency bands • Masking and Temporal Masking are used Fast Forward Your Development www.dsp-ip.com
  • 15. IMAGE COMPRESSION Fast Forward Your Development www.dsp-ip.com
  • 16. What is a digital image? • Audio PCM ▫ One 1-D array of sample • BMP Image ▫ Three 2-D arrays of 2D numbers representing Red, Green and Blue values Fast Forward Your Development www.dsp-ip.com
  • 17. Image Compression? Why? • Image size = 720*580 720 580 • 3 Image Layers RGB =720*580*3 • 8 Bits per pixel 720*580*3*8 720 580 3 8 = 10022400 bits • L t of bit f one L Lots f bits for Lena Fast Forward Your Development www.dsp-ip.com
  • 18. Color matching • How can we compare colors so that the content creators and consumers know what kno hat they are seeing? •MMany diff different ways t including CIE chromacity di h it diagram Fast Forward Your Development www.dsp-ip.com
  • 19. Video Color Transforms • Largely derived from older analog methods of coding color for TV. Luminance is separated from color TV information. • YIQ is used to transmit TV signals in North America and Japan.This coding also makes its way into VHS video tape coding in these countries since video tape technologies also use YIQ. • In Europe, video tape uses the PAL or SECAM codings, which are based on TV that uses a matrix transform hi h b d h i f called YUV. • Finally digital video mostly uses a matrix transform Finally, called YCbCr that is closely related to YUV Fast Forward Your Development www.dsp-ip.com
  • 20. YUV Separation Fast Forward Your Development www.dsp-ip.com
  • 21. YUV Color Model •YUV codes a luminance signal (for gamma-corrected signals) equal to Y , the “luma". •Chrominance refers to the difference between a color and a reference white at the same luminance. (U and V) The transform is: Fast Forward Your Development www.dsp-ip.com
  • 22. RGB->YUV Color Transform G G B B Y U V R R Fast Forward Your Development www.dsp-ip.com
  • 23. YIQ Color Model YIQ is used in NTSC color TV broadcasting. broadcasting Again, gray pixels generate zero (I;Q) chrominance signal. g I and Q are a rotated version of U and V . The transform is: Th t f i Fast Forward Your Development www.dsp-ip.com
  • 24. YCbCr Color Model 1. The Rec. 601 standard for digital video uses another color space YCbCr which closely related to the YUV transform. 2. The YCbCr transform is used in JPEG image compression and MPEG video compression. For 8-bit coding: Fast Forward Your Development www.dsp-ip.com
  • 25. Sampling p g • Digitization of analog signal • Voice - Sampling in time only The number of only. samples per second is the Frequency – F. in telephony, telephony voice sampling frequency is 8KHz to transfer 4KHz of sound. •IImage - SSampling in 2D space - F F O thi li i Fx, Fy. On this screen the images are sampled 1024x768 • Vid - S Video Sampling i 2D space and ti li in D d time – F F Fx, Fy, Fn. On TV Fn is 25 or 30 fps (frames per second) Fast Forward Your Development www.dsp-ip.com
  • 26. Quantization • Lowering the number of bits needed to describe an object object. • For example I scored 66 in the test an object with 100 steps need 7 description bits bits. • How is she looking ? 10 ! Needs only 5 bits for description. d i ti • Could we lower the number of bits more? What is th i the minimum? i i ? • How to describe an object with one bit? Fast Forward Your Development www.dsp-ip.com
  • 27. Color based decimation • Our eyes have better resolution and scaling for luminance then for color color. • Compress color by using 4:2:0 method Fast Forward Your Development www.dsp-ip.com
  • 28. Counting the bits • How much can we save by color compression? ▫ 3*Image size in RGB 24 bit color representation representation. ▫ 1 + 2*1/4 Image size in 4:2:0 YUV representation. ▫ Compression ratio is 2 !! • Actual saving is bigger due to different Y and UV quantization. quantization Fast Forward Your Development www.dsp-ip.com
  • 29. Linear Transform • If the signal is formatted as a g • Energy compaction p p y gy p property: vector, a linear transform can The transformed signal vector be formulated as a matrix- has few, large coefficients and vector product that transform many nearly zero small the signal into a different coefficients. These few large domain. coefficients can be encoded • Examples: efficiently with few bits while ▫ K-L Transform retaining the majority of ▫ Discrete Fourier Transform energy of the original signal. ▫ Discrete cosine transform ▫ Discrete wavelet transform Fast Forward Your Development www.dsp-ip.com
  • 30. Block-based Image Coding • Block-based image Block based • Advantages: coding scheme: ▫ Parallel processing can partitions the entire be applied to p pp process image into 8 by 8 or individual blocks in parallel. 16 by 16 (or other ▫ Redundant information size) blocks. in close proximity (like • The coding algorithm cache) is applied to individual blocks independently. Fast Forward Your Development www.dsp-ip.com
  • 31. Transform - DCT • The DCT transform the data from pixel intensity to frequency intensity intensity. • Low frequency are important high frequency less 1 7 7 (2m  1)u (2n  1)v • (You’ll F (u, v) cos even if you don’t remember the  4 get launch  u 0 v 0 16 cos 16 m  n  0; f (m, n)   7 7 IDCT1f  formula above) 1)u (2n  1)v l b (2m)  8  F (u , v) cos cos 0  m, n  7; m  n  0.  u 0 v 0 16 16 Fast Forward Your Development www.dsp-ip.com
  • 32. DCT Coefficients Quantization Fast Forward Your Development www.dsp-ip.com
  • 33. AC Coefficients • AC coefficients are first weighted with a quantization i h d ih i i 1 2 6 7 15 16 28 29 matrix: 3 5 8 14 17 27 30 43 C(i,j)/q(i,j) C(i j)/q(i j) = Cq(i j) (i,j) 4 9 13 18 26 31 42 44 Then quantized. 10 12 19 25 32 41 45 54 • Then they are scanned in a 11 20 24 33 40 46 53 55 zig-zag order into a 1D 21 23 34 39 47 52 56 61 sequence to be subject to AC 22 35 38 48 51 57 60 62 Huffman encoding. ff di 36 37 49 50 58 59 63 64 • Question: Given a 8 by 8 array, array how to convert it into a Zig-Zag scan order vector according to the zig- zag scan order? What is the g algorithm? Fast Forward Your Development www.dsp-ip.com
  • 34. DCT Basis Functions Fast Forward Your Development www.dsp-ip.com
  • 35. DCT compression Example Original Image Fast Forward Your Development www.dsp-ip.com
  • 36. DCT 1 coefficient Fast Forward Your Development www.dsp-ip.com
  • 37. DCT 6 coefficients Fast Forward Your Development www.dsp-ip.com
  • 38. DCT 20 coefficient Fast Forward Your Development www.dsp-ip.com
  • 39. JPEG Image Coding Algorithms Q Quantization DC 8x8 Matrix DC DPCM Huffman block DCT Q Zig Zag AC AC Scan Huffman Code books JPEG Encoding Process Fast Forward Your Development www.dsp-ip.com
  • 40. Generalization of JPEG Coding Transform Entropy Color, Color Frequency Quantize Reorder Coding JPEG Encoding Process Fast Forward Your Development www.dsp-ip.com
  • 41. Video Coding Basics Fast Forward Your Development www.dsp-ip.com
  • 42. Video Coding • Video coding is often implemented as encoding a sequence of i f images.Motion compensation i i is used to exploit temporal redundancy between successive f i frames. • Examples: MPEG-I, MPEG-II, MPEG-IV, H.263, H.263+, H264 • Existing video coding standards are based on JPEG image compression as well as motion compensation. Fast Forward Your Development www.dsp-ip.com
  • 43. Video Coding Standardization Scope • Only restrictions on the Bitstream, Syntax, and Decoder are standardized: ▫ Permits the optimization of encoding ▫ Permits complexity reduction p y ▫ Provides no guarantees on quality Fast Forward Your Development www.dsp-ip.com
  • 44. Video Encoding Buffer control Current frame x(t) r Bit stream + DCT Q VLC Buffer  Q-1 This is a simplified block diagram where the encoding of intra coded IDCT frames is not shown. Xp(t): predicted ^ reconstructed residue r(t): frame + ^(t) x(t): reconstructed t t d Motion ^ current frame x(t) x(t-1) Frame Estimation & Compensation Buffer Motion vectors Fast Forward Your Development www.dsp-ip.com
  • 45. Video Encoding Color Frequency Transform Buffer control Transform + Q Reorder Entropy  Q-1 This is a simplified block diagram where the encoding of intra coded Tf-1 frames is not shown. Xp(t): predicted ^ reconstructed residue r(t): frame + ^(t) x(t): reconstructed t t d Motion ^ current frame x(t) x(t-1) Frame Estimation & Compensation Buffer Motion vectors Fast Forward Your Development www.dsp-ip.com
  • 46. Forward Motion Estimation 1 2 3 4 1 2 4 3 5 6 7 8 5 7 8 6 9 10 11 12 9 11 12 10 0 13 15 16 13 14 15 16 14 Current frame constructed From different parts of reference frame Reference frame Fast Forward Your Development www.dsp-ip.com
  • 47. Video sequence : Tennis frame 0, 1 previous frame current frame 50 50 100 100 150 150 200 200 50 100 150 200 250 300 350 50 100 150 200 250 300 350 Fast Forward Your Development www.dsp-ip.com
  • 48. Frame Difference Frame Difference :frame 0 and 1 Fast Forward Your Development www.dsp-ip.com
  • 49. What is motion estimation? Motion Vector Field of frame 1 50 0 -50 -100 -150 -200 -250 0 50 100 150 200 250 300 350 400 Fast Forward Your Development www.dsp-ip.com
  • 50. What is motion compensation ? Motion compensated frame 50 100 150 200 50 100 150 200 250 300 350 Fast Forward Your Development www.dsp-ip.com
  • 51. Motion Compensated Frame Difference p Motion Compensated Frame Difference :frame 0 and 1 Frame Difference :frame 0 and 1 Fast Forward Your Development www.dsp-ip.com
  • 52. Frame Types • Three types of frames: yp ▫ Intra (I): the frame is coded as if it is an image ▫ Predicted (P): predicted from an I or P frame ▫ Bi-directional (B): forward and backward predicted from a pair of I or P frames. • A typical f i l frame arrangement i is: I1 B1 B2 P1 B3 B4 P2 B5 B6 I2 • P1, P2 are b h f both forward-predicted f d di d from I1. B1, B2 are interpolated from I1 and P1, B3, B4 are interpolated from P1, P2, and B5, B6 are interpolated from P2, I2. • New Coding standards added other frame types: SP, SI, D Fast Forward Your Development www.dsp-ip.com
  • 53. Macro-blocks and Blocks Y(16x16) Cr (8x8) RGB Cb (8x8) 16x16x3 Fast Forward Your Development www.dsp-ip.com